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Introduction 


Shall we be destined to the days of eternity, on holy-days,as well as 
working days, to be shewing the RELICKS OF LEARNING, as monks do 
the relicks of their saints — without working one — one single miracle 
with them? 

Laurence Sterne, Tristram Shandy 


This book deals with information processing; so it is far from being a book 
on information theory (which would be built on description and estimation). 
The reader will be shown the horse, but not the saddle. At any rate, at the 
very beginning, there was a series of lectures on “Information theory, through 
the looking-glass of an algebraist”, and, as years went on, a steady process of 
teaching and learning made the material evolve into the present form. There 
still remains an algebraic main theme: algorithms intertwining polynomial 
algebra and matrix algebra, in the shelter of signal theory. 

A solid knowledge of elementary arithmetic and Linear Algebra will be the 
key to a thorough understanding of all the algorithms working in the various 
bit-stream landscapes we shall encounter. This priority of algebra will be the 
thesis that we shall defend. More concretely: We shall treat, in five chapters 
of increasing difficulty, five sensibly different subjects in Discrete Mathemat- 
ics. The first two chapters on data compaction (lossless data compression) and 
cryptography are on an undergraduate level — the most difficult mathematical 
prerequisite will be a sound understanding of quotient rings, especially of fi- 
nite fields (mostly in characteristic 2). The next two chapters are already on a 
graduate level; the reader should be slightly acquainted with arguments in sig- 
nal theory — although Lebesque integration could remain the “grey box” that 
it usually is. We encounter sampling — an innocent operation of tremendous 
epistemological impact: the Platonic mathematician leaving his heaven of con- 
tinuity (rule = truth) for the earth of discreteness (diversity = uncertainty) will 
be plainly comforted by the great interpolation theorems that lift him back to 
the heights. The chapter on error control codes which are designed according 
to signal theoretical ideas, complements — on a purely algebraic level — the 
invasion of signal theory. The fifth and final chapter is the most important, 
in length as well as in complexity. It deals with lossy (image) compression, 
and yields the mathematical background for the understanding of JPEG and 
JPEG 2000. Now, our Platonic mathematician will be expelled from paradise: 
The discrete world becomes absolute, and all continuous constructions are 
plainly auxiliary and relative. 

But let us pass to a detailed description of the content. 

The first chapter on data compaction is more or less an elementary intro- 
duction to algorithmic information theory. The central theme will be the non- 
redundant representation of information. Everything turns around the notion 
of entropy: What is the information content of a string of symbols (with given 
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statistical behaviour), i.e. what is its minimal bit equivalent ? Entropy coding 
has its algorithmic stars: for memoryless sources, Huffman entropy coding is 
unbeatable, but from a dynamic viewpoint, arithmetic coding will be slightly 
better. Both methods are plainly integrated in advanced image compression 
standards — we shall give a “default” Huffman table for JPEG. The chapter 
will end with an (merely descriptive) exposition of the algorithm LZW which 
is universal in the sense that it compacts any character stream — without pre- 
liminary statistical evaluation — by establishing a dictionary that enumerates 
typical substrings (thereby creating its proper statistical evaluation). LZW is 
the perfect data compaction algorithm — but it needs large files in order to be 
efficient. That is why we do not meet it in image compression where the data 
units are too small. 

The second chapter presents a set of rather austere lectures on cryptogra- 
phy. We aim to give the maximum of information in a minimum of space — 
there already exists a lot of highly coloured frescoes on the subject in print. 
The venerable algorithm DES — the cryptosystem the best understood on 
planet earth — will serve as an introduction to the subject. Things become 
more serious with the new standard AES-Rijndael, the mathematical base- 
ment of which is a nice challenge to the student’s understanding of higher level 
(still) elementary arithmetic. He will learn to think in cyclic arithmetic — thus 
getting familiar with discrete logarithms in a very explicit way. This opens 
the door to digital signatures, i.e. to the practical realization of the public key 
paradigm: I tell you my position on an arithmetic circle, but I do not reveal 
the number of steps to get there. We shall treat the principal standard for 
digital signatures, the system DSA (Digital Signature Algorithm), as well as 
the variants rDSA (signatures via RSA) and ECDSA (signatures via elliptic 
curve arithmetic). As to RSA: This thirty-year-old algorithm has always been 
the cornerstone of academic zest to promote the public key idea. So we shall 
follow tradition — not without noting that RSA is a little bit old fashioned. 
Finally, the secure hash algorithm (SHA-1) will produce the message digests 
used in the various signature protocols. We shall need a lot of large prime 
numbers; hence we include a brief discussion on their efficient generation. 

This completes the description of the easy part of this book. Teaching expe- 
rience shows that students like data compaction for its simple elegance and are 
distant towards the iterative flatness of most cryptographic standards — are 
they to blame? 

With the third chapter, we enter the mathematical world of signal theory. 
We have to answer the question: What is the discrete skeleton of a (con- 
tinuous) signal? This means sampling, and reconstruction via interpolation. 
Putting aside all practical considerations, we shall treat the problem in vitro. 
Tough mathematical expositions are available; we have chosen a step-by-step 
approach. So, we begin with the discrete Fourier transform and its importance 
for trigonometric interpolation. Then we show ad hoc the classical interpola- 
tion theorem (of Whittaker-Shannon, Nyquist-Shannon, or simply Shannon, 
as you like it...) for precisely trigonometric polynomials. Finally, we attack 
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the interpolation theorem in its usual form. There are some formal prob- 
lems which need a short commentary. The natural mathematical framework 
for signal theory is the L? Hilbert space formalism. Now, elements of an L? 
space are not functions (which disappear in their clouds of equivalence) but 
function behaviour sketches. Precise numerical rules enter via duality. Thus, 
sampling — which is basically a Hilbert space nonsense — must be considered 
as a rule of behaviour (and should be duly formalized by a distribution). The 
equality in the Shannon interpolation formula (which means equality of dis- 
tributions) is, in any down-to-earth exposition, considerably fragilized by the 
proof that establishes it. We shall try to be as simple as possible, and avoid 
easy “distribution tricks” . 

Logically, it is the fifth and last chapter on data compression that should 
now follow. Why this strange detour in the land of error control codes? There 
are at least two reasons. First, we get an equilibrium of complementary lec- 
tures, when alternating between non-algebraic and algebraic themes. Then, 
the fourth chapter logically reinforces our definite submission to signal theory. 
The codes of Reed-Solomon — our first subject — have a nice error- correcting 
algorithm that makes use of the Discrete Fourier Transform over finite fields 
of characteristic 2. And the convolutional codes — our second subject — are 
best understood via digital filtering in binary arithmetic. Our exposition there 
is non-standard, with a neat accent on algorithmic questions (no trellis nor 
finite automata formalisms). 

Finally, we come to the fifth chapter, which is rather voluminous and treats 
data compression, i.e. the practice of intentionally reducing the information 
content of a data record — and this in such a way that the reproduction has 
as little distortion as possible. We shall concentrate on image compression, in 
particular on JPEG and JPEG 2000. The quality of compression depends on 
sifting out efficiently what is considered to be significant numerical informa- 
tion. Quantization towards bit representation will then annihilate everything 
that can be neglected. 

Our main concern will be to find an intelligent information theoretic sieve 
method. 

It is the Discrete Cosine Transform (DCT) in JPEG, and the Discrete 
Wavelet Transform (DWT) in JPEG 2000 that will resolve our problems. 
In both cases, a linear transformation will associate with regions of digital 
image samples (considered as matrices of pictural meaning) matrix transforms 
whose coefficients have no longer a pictorial but only a descriptive meaning. 
We must insist: Our transformations will not compress anything; they merely 
will arrange the numerical data in a transparent way, thus making it possible 
to define sound quantization criteria for efficient suppression of secondary 
numerical information. 

We shall begin the fifth chapter with a slight non-thematic digression: 
the design of digital passband filters in a purely periodic context. This will 
be a sort of exercise for formally correct thinking in the sequel. Then we 
come up with the discrete cosine transform and its raison d’étre in JPEG. 
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We shall first treat the 1D DCT (acting on vectors), then its 2D extension 
(acting on matrices), and finally its position in the Karhunen—Loéve family. 
For the pragmatic reader who is only interested in image compression there 
is an easy argument in favour of the DCT that short circuits everything: 
The 2D DCT acts via conjugation on 8 x 8 matrices, preserves the energy (the 
Euclidian norm of matrices), and diagonalizes all constant matrices (which is 
a reasonable motive in image compression). 

In the second part of the last chapter we shall encounter the discrete 
wavelet transform and its implantation into JPEG 2000. Exactly like the 
Karhunen—Loéve transform, the discrete wavelet transform is rather an en- 
tire family of transforms. Our presentation will adopt the (two channel) filter 
bank approach, which is easy to explain — and plainly sufficient for the un- 
derstanding of the way discrete wavelet transforms act in image compression. 
More concretely, we shall concentrate on three specific wavelet transforms: the 
DWT 5/3 spline, the DWT 7/5 Burt and the DWT 9/7 CDF. We shall also 
treat the reversible mode of JPEG 2000: how to get invertible transformations 
in integer arithmetic. The “lifting structure” that will guarantee non-linear 
integer-valued approximations of our initial matrix transformations bears a 
clear resemblance to the sequence of round transforms of the cryptosystem 
DES. 

At the very end of our book we have to answer the question: Where are the 
wavelets (and why are there wavelets) behind all of that filter bank theory? 
This is a pretty mathematical subject — maybe a little bit too mathematical. 
But a thorough understanding of the criteria that govern the design of filter 
banks requires adopting the wavelet viewpoint. 

Let us end this introduction with some remarks on teaching questions. This 
is a book on Mathematics. What about proofs? We have adopted a strategy 
that conciliates aesthetics with common sense: A proof should be interesting, 
not too long, and it should give sufficient information on the mathematical 
weight of the proposition. For example, the proof of the Kraft inequality (char- 
acterizing prefix codes) is of this kind. On a quite different level of reasoning, 
almost all proofs around the Viterbi decoding algorithm (for convolutional 
codes) are welcome, since they do not abound in common literature. In a cer- 
tain sense, it is precisely our presentation of wavelet theory that shows the 
“sieve of rigour” that we have adopted. 

A short remark on the nature of our exercises: Some readers will — 
perhaps — be shocked: Almost all exercises are dull, rather mechanical and 
lengthy. But we learn by repetition... Towards the end of the book, a good 
pocket calculator will be necessary. At any rate, a book in Concrete Mathe- 
matics should inflict the burden of complexity in a quantitative, rather than 
in a qualitative way. We have given a lot of hints, and many solutions. But 
we never aimed at completeness... 

Let me end with a rather personal remark. This book is a result of a happy 
conjunction of teaching and learning. Learning has always been exciting for 
me. I hope the reader will feel it the same. 


1 


Data Compaction 


This first, rather elementary chapter deals with non-redundant representation 
of information; in other words, we shall treat data compaction codes (i.e. 
algorithms for lossless data compression). More common, eventually lossy, 
data compression needs arguments and methods from signal theory, and will 
be considered in the last chapter of this book. 


1.1 Entropy Coding 


All coding methods that we shall encounter in this section are based on a 
preliminary statistical evaluation of our set of data. In a certain sense, the 
coding algorithms will treat the statistical profile of the data set rather than 
the data itself. Since we are only interested in coding methods, we shall always 
feel free to assume that the statistics we need are plainly at our disposal — so 
that our algorithms will run correctly. 

Note that our probabilistic language is part of the tradition of informa- 
tion theory — which has always been considered as a peripheral discipline 
of probability theory. But you are perfectly allowed to think and argue in a 
purely deterministic way: the statistical evaluation of the data for compaction 
can be thought of as a specification of parameters — in the same way as the 
choice of the right number of nodes (or of the correct sampling frequency) in 
interpolation theory. 

A historical remark: do not forget that almost all good ideas and clever 
constructions in this section have come to light between 1948 and 1952. 


1.1.1 Discrete Sources and Their Entropy 


We shall consider memoryless discrete sources, producing words (strings of 
letters, of symbols, of characters) in an alphabet {ao,a1,...,an—1} of N 
symbols. 
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We shall call 
pj = p(a;) = the probability of (the production of) the letter a;,0 <j < 
N-1. 


Notation p = (po,pi,---;PN—1) = the probability distribution which de- 
scribes the production of our source. 


— Regarding the alphabet: think of {0,1} (a binary source: for example, a bi- 
nary facsimile image) or of {00000000, 00000001,..., 11111111} (a source 
of 256 symbols, in 8-bit byte representation: for example, the ASCII char- 
acter code). 

— Regarding the memoryless production: this is a condition of probabilistic 
modelling which is very strong. Namely: 


For a word w = aj,aj;,---a;, of length n, the statistically independent pro- 
duction of its letters at any moment is expressed by the identity 


P(W) = p45, )P(Aj2) +++ (45). 


This identity (the probability of a word is the product of the probabilities of 
its letters) models the production of our source by the iterated roll of a loaded 
dice, the faces of which are the letters of our alphabet — with the probability 
distribution p describing the outcome of our experience. 

Note that this rather simple modelling has its virtues beyond simplicity: 
it describes the ugliest situation for data compression (which should improve 
according to the degree of correlation in the production of our data), thus 
meeting the demands of an austere and cautious design. 

At any rate, we now dispose of an easy control for modelling — having a 
sort of “commutation rule” that permits us to decide what should be a letter, 
i.e. an atom of our alphabet. For a given binary source, for example, the words 
01 and 10 may have sensibly different frequencies. It is evident that this source 
cannot be considered as a memoryless source for the alphabet {0,1}; but a 
deeper statistical evaluation may show that we are permitted to consider it 
aS a memoryless source over the alphabet of 8-bit bytes. 


The Entropy of a Source 


The entropy of a (discrete) source will be the average information content of 
a “generic” symbol produced by the source (measured in bits per symbol). 

Let us insist on the practical philosophy behind this notion: you should 
think of entropy as a scaling factor towards (minimal) bit-representation: 1,000 
symbols produced by the source (according to the statistics) “are worth” 
1,000 x entropy bits. 


Prelude The information content of a message. 


Let I(w) be the quantity of information contained in a word w that is pro- 
duced by our source. We search a definition for I(w) which satisfies the 
following two conditions: 
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(1) I(w) is inversely proportional to the probability p(w) of the production 
of w (“the less it is frequent, the more it is interesting”). 
Moreover, we want the information content of a sure event to be zero. 
(2) I(aj,@j.-°--a;,) = I(a;,) + 1(aj,) +--+ + I(a;,) (the information content 
of a word is the sum of the information contents of its letters — this stems 
from our hypothesis on the statistical independence in the production of 
the letters). 


Passing to the synthesis of (1) and (2), we arrive at the following condition: 


I(w) = F(—1.) where the real function F has to be strictly monotone 
p(w) 


(increasing) and must satisfy the identity F(a-y) = F(x) + F(y) as well as 
F(1) =0. 

Now, there is essentially only one (continuous) function F' which satisfies 
our conditions: the logarithm. 

Thus the following definition comes up naturally. 


Definition I(w) = Log, (xs) = —Logpp(w). 


eyln2 Lnz 


[ Recall: y = Logox u= 29 x 
But why the logarithm to the base 2? 


Answer We want the unity of the information content to be the bit. 
Let us make things clearer with two examples. 


(a) Consider a source which produces a9 = heads and a, = tails with the 
same probability (p = (po,pi) = (5.4)). We get I(a9) = I(ai) = 
—Log,271 = 1. 

That is logical: when tossing coins with equal chance, heads is naturally 
coded by 0 and tails ts naturally coded by 1. 

(b) Let us pursue this line of thought: now, our outcome will be the 256 in- 

tegers between 0 and 255 (rolling a very, very big dice with 256 faces), 


all of equal chance: p = (po, P1,---,P255) = (sa: oat seen =) . I(ao) = 
I(ay) =+++ = I(az55) = —Log,2~° = 8. Once more: no surprise; assuming 
equal chance, the information content of any of the integers 0,1,...,255 


has to be 8 bits: they are 8-bit bytes! 
But back to our source: 


Let p = (po, P1,---;PN-—1) be the probability distribution which describes 
the (memoryless) production of the letters of our alphabet. 


Definition of the entropy of the source: 


H(p) = the average quantity of information per symbol (in bits per symbol) 
= polo + pili +--+ + pw—1ln-1 = —poLogspo — piLogspi 
er ey — pn—1LOogopn-1.- 
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(1) Compute the entropy of the source which produces the eight letters 
ao, @1,---,@7, according to the probability distribution p = (po, p1,---,p7) 
with po = 5,71 = 9, P2 = P3 = 7g.P4 = Ps = Po = P7 = 55- 

(2) Let us consider a memoryless source which produces four letters 
a, 41, 42, 43, according to the probability distribution p = (po, p1, p2, p3). 
Let us change our viewpoint. Consider the source as a producer of the 16 
symbols apag, @9a1,...,@342, 4343, according to the product distribution 
p®) = (poo, Por, «+. P23, P33) with piz = pipj, 0< i,j <3. 

Show that H(p°)) = 2H(p). Generalize. 


Remarks 


Our situation: the alphabet {ao, a1,...,@n—1} will remain fixed; we shall vary 
the probability distributions. . . 


f H(p)=0 <> The source produces effectively only one letter 
) (for example the letter ao), with p(ag) = 1. 

Recall: a sure event has information content zero. 

Hence: the entropy will be minimal (will be zero) as a characteristic of a 
constant source production. 

Thus, we extrapolate (and we are right): 

(2) H(p) is maximal Po = Pi ="** =Pn-1 We 

In this case, we have H(p) = Log, N. 


Exercises 


(1) A binary source produces ag = white and a; = black according to the 
probability distribution p = (po, p1). 
Find the condition on the ratio white/black which characterizes H(p) < 4. 
(2) Gibbs’ inequality. 
Consider p = (po,Pi,---,PN—1) and q = (qo,%,;---;GN—1), two strictly 
positive probability distributions (no probability value is zero). 
(a) Show that — oe pj Logsp; < — a. Pj LOS 2q)- 
(b) Show that the inequality above is an equality <> p = q. 
(c) Deduce from (b): every probability distribution p = (po, p1,..-,PN-1) 
satisfies H(p) < Log,N with equality po = Pi =+': = PN-1 
1 


Ne 
Hint: 


(a) Recall: Luna < x —1 for all x > 0 with equality => ¢ =1. 
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(b) You should get from (a) the following inequality 


N-1 : : 
So (et (2-9) <a 
am; Dj Pj 


where all the terms of the sum are non-positive. This is the clue. 


Entropy Coding, A First Approach 


Consider a memoryless source which produces the N symbols ag, a1,...,@N—1, 
according to the probability distribution p = (po, p1,-.--,PN—1)- 

We have seen: every letter a; “is worth” I(a;) bits, 0 <7 < N—-1. 
This leads to the natural idea (Shannon (1948)): associate with the symbols 
of our alphabet binary code words of variable length in such a way that the 
length of a code word associated with a letter is precisely the information 
content of this letter (assume first that all probabilities are powers of 2, so 
that the information contents will be correctly integers). 

More precisely: 

Let 1; be the length (the number of bits) of the code word associated to 
the letter aj,0 <7 < N-1. 

Our choice: |; = I(a;),O0<j<N-1. 

Let us look at the average length | of the code words: 


l= polo + pili +++: +pn-iln-1. 


Note that / is a scaling factor: our encoder will transform 1,000 symbols pro- 
duced by the source (in conformity with the statistics used for the construction 
of the code) into 1,000 x I bits. 

But, since we were able to choose |; = I(a;), 0 <j < N —1, we shall get 


l= H(p). 


Example Recall the source which produces the eight letters ao, a1,...,@7, ac- 
cording to the probability distribution p = (po,p1,...,p7) with pp = $,p1 = 
i> P2 P3 75» PA P5 = P6 = P7 mor 

This means: I(ao) = 1,2 (a1) = 2,2 (a2) = I(a3) = 4,2 (a4) = I(as) = 
I(ag) = I(az) = 5. 

We choose the following encoding: 


aor OO aqgtr— 11100 
ay’ 10 as+— 11101 
a2 '*— 1100 ag + 11110 
a3+— 1101 a7+— 11111 


Encoding without statistics, i.e. assuming equal chance, will oblige us to re- 
serve three bits for any of the eight letters. On the other hand, with our code, 
we obtain |! = H(p) = 2.125. 
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Let us insist: without statistical evaluation, 10,000 source symbols have to 
be transformed into 30,000 bits. With our encoder, based on the statistics of 
the source, we will transform 10,000 letters (produced in conformity with the 
statistics) into 21,250 bits. Manifestly, we have compressed. 


Important remark concerning the choice of the code words in the ex- 
ample above. 

Inspecting the list of our eight code words, we note that no code word is 
the prefix of another code word. We have constructed what is called a binary 
prefix code. In order to understand the practical importance of this notion, let 
us look at the following example: 

Ar->0 Br-dl Cr 10. 

Let us try to decode 001010. We realize that there are three possibilities: 
AACC, ABAC, ABBA. The ambiguity of the decoding comes from the fact 
that the code word for A is the prefix of the code word for B. But look at our 
example: there is no problem to decode 

01101001110111111110001000 back to aga3apaga5a7a24941a9a0. 


McMillan (1956) has shown that every variable length binary code that 
admits a unique decoding algorithm is isomorphic to a prefix code. This will 
be the reason for our loyalty to prefix codes in the sequel. 


1.1.2 Towards Huffman Coding 


In this section we shall recount the first explosion of ideas in information 
theory, between 1948 and 1952. Everything will begin with Claude Shannon, 
the founder of the theory, and will finally attain its “price of elegance” with 
the algorithm of Huffman, in 1952. 

Do not forget that the theory we shall expose is built upon the rather 
restrictive hypothesis of a memoryless source.! 


The Kraft Inequality and its Consequences 


Let us consider a memoryless source producing N letters, ao,a1,...,@N-1, 
according to the probability distribution p = (po, p1,..-,DN—1). 


Shannon’s coding paradigm. Associate to ap, a1,-...,@N—1 words of a binary 
code, such that the lengths lo,1,,...,ly_—1, of the code words will correspond 
to the information contents of the encoded symbols. 


We need to make precise the term “will correspond to the information 
contents of the encoded symbols”. 
We aim at 
l; = I(a;) = —Logyp;, O<j<N-1. 


' One can do better — but there are convincing practical arguments for simple 
modelling. 
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More precisely, we put 
1; = [I(a3)] = [—Loggp;], OS F< N-1, 


where | | means rounding up to the next integer. 

Our first problem will now be the following: 

Is Shannon’s programme soundly formulated: Suppose that we impose N 
lengths Ig, 11,...,/—1 for the words of a binary code to be constructed. What 
are the conditions that guarantee the existence of a binary prefix code which 
realizes these lengths? In particular, what about the soundness of the list of 
lengths derived from a probability distribution, following Shannon’s idea? Is 
this list always realizable by the words of a binary prefix code? 

Let us write down most explicitly the Shannon-conditions: 


Ll; —1 < —Logyp; < 1,, 0<j<N-1, i.e. 


209 opps 22 2's O0<j<N-1. 


Summing over all terms, we get: 


N-1 
1 1 1 
ly 1 eee 
» 2 ~ 9lo * Oh Fe " 9ln—1 <1 
j=0 


This innocent inequality will finally resolve all our problems. 


We begin with a (purely combinatorial) result that has gloriously survived 
of a dissertation published in 1949: 


Proposition (Kraft’s Inequality) Let Io,h,...,ly—1 be imposed lengths 
(for N binary code words to construct). Then the following holds: 

There exists a binary prefix code which realizes these lengths <> 
Dja-0 24 <1. 


Proof Consider the binary tree of all binary words: 


0 1 
00 01 10 11 


000 O01 O10 O11 100 101 110 = 111 


On level 1, there are 2! binary words of length 1, arranged according to their 
numerical values (every word, considered as the binary notation of an integer, 
indicates its position). The successors of a word (for the binary tree structure) 
are precisely its syntactical successors (i.e. the words which admit our word 
as a prefix). 

This will be the convenient framework for the proof of our claim. 

= =: Choose 1 > 1;,0 < 3 < N —1. Every word of length /; has 
2'—“i successors on the level | of the binary tree of all binary words. The pre- 
fix property implies that these level-/-successor sets are all mutually disjoint. 
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Comparing the cardinality of their union with the number of all words on 
N-1 o5I_U, : ane nae 
level 1, we get: )7 5 gil! ae: Vj=0 2 ee Ny 
<=: Put | = max{l; : 0 < 7 < N— 1}, and let n1,72,...,m be the 
numbers of code words of length 1,2,...,! that we would like to construct. 
By our hypothesis we have: 


Ue OF Gig Br ee ts Oo 4, 


i.e. 
ny: 2-1 <1 ny <2 
ny-2-!+ng-2-2 <1 Ng <22—n,-2 
ny 27) + ng 277 eee tn 2) <1 ny < 2! — ny, - 21 = 2--— yy 2 


The first inequality shows that we can make our choice on level 1 of the binary 
tree of all binary words. The second inequality shows that the choice on level 
2 is possible, after blockade of the n, - 2 successors of the choice on level 1. 
And so on... 

The last inequality shows that the choice on level / is possible, after block- 
ade of the n;-2'~1 successors of the choice on level 1, of the nz-2'~? successors 
of the choice on level 2,..., of the n;-2 successors of the choice on level | — 1. 

This finishes the proof of our proposition. 


Exercises 


(1) You would like to construct a binary prefix code with four words of length 
3, and six words of length 4. How many words of length 5 can you add? 

(2) Consider an alphabet of four letters: N, E, S, W. 
Does there exist a prefix code on this alphabet which consists of two words 
of length 1, four words of length 2, 10 words of length 3 and 16 words of 
length 4? 

(3) A memoryless source produces eight letters A, B, C, D, E, F, G, H ac- 
cording to the probability distribution p = (p(A), p(B),...,p(H)), with 


(a) Determine the information content of every letter and compute the 
entropy H(p) of the source. 

(b) Following Shannon’s coding paradigm, find a binary prefix code asso- 
ciated with p. 

(c) Compute the average length / of the code words, and compare it with 
A(p). 
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The most important consequence of the characterization of prefix codes via 
Kraft’s inequality is the following theorem. Its small talk version could be: 
There is no lossless compression below entropy. 


Theorem Consider a memoryless source which produces N letters ag, aj,... 
an-—1 according to the probability distribution p = (po, p1,---,PN-—1)- 
Let C be some associated binary prefix code, and 


the average length of the code words (in bits per symbol). 

Then: H(p) <l. 

Moreover, the binary prefix codes constructed according to Shannon’s idea 
satisfy the following inequality: | < H(p) +1. 


Proof (1) H(p) —1 <0: 

7 N-1 N-1 N-1 <j 
H(p)—l= ap are, pj Logsp; — D6 pjlj = iad are pjLn (= ) 
Now: Lux < x —1 for x > 0, hence 

7 N-1 ey N-1 £23 
H(p)-I< Tao a0 Pj (2 - 1) = raven ee (2-4 — pj). 


js (2-44 — pj) < 0, and we 


But, due to Kraft’s inequality, we have: > 
are done. 

(2) Recall: following Shannon’s idea, one gets for the lengths of the code words 
associated with our symbols: 


Ll; —1< —Logsp; <1), O0<j<N-1. 


Summing up yields: er (pjl; — pj) < — oe pj; Logyp,, i.e. 
L< H(p)+1. 


Shannon Codes 


Shannon coding is precisely the algorithmic realization of Shannon’s coding 
paradigm: 

Encode every source symbol into a binary word — the length of which 
equals the information content of the source symbol (rounded up to the next 
integer). We will obtain binary prefix codes. Unfortunately, Shannon codes 
are not always optimal (in a natural sense, which shall be made precise later) 
and were soon dethroned by Huffman coding. Why shall we altogether dwell 
on Shannon coding? 

The principal reason is that arithmetic coding which is a very interesting 
“continuous” method of compaction (integrated in certain modes of JPEG 
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and, more expressly, of JPEG 2000) is nothing but a dynamic version of Shan- 
non coding. With the Shannon codes we are in the antechamber of arithmetic 
coding. 

The idea of Shannon’s algorithm is the following: 

Consider a memoryless source producing N letters ao,a1,...,@N—1, ac- 
cording to the probability distribution p = (po, p1,.--,;PN—1)- 

Assume po > pi > ++: > pn—1 (in order to guarantee that the following 
constructions will yield a prefix code). Associate with p a partition of the 
interval (0, 1[ in the following way: 


Ay =0, 
Aj = Po, 
Ag =pot+?pi, 


A3 =potpit po, 


An =potpit:::+pn-1=1. 


We note that, the length of every interval [A,;, A;+1[ equals the probability 
of (the production of) the letter a;: 
py =Ajzi— Aj, OSG SN-1. 


We shall associate with the letter a; a binary word c;, which will be code word 
of the interval [A;, Aj+1[: 


G = CA, Ana), 07 2N —1, 


Let us point out that the realization of Shannon’s program demands that the 
length of the jth code word should be: 


1; = [I(aj)] = [—Logap;] = [—Loge(Aj+1 — 4j)], OS FS N-1. 


These considerations will oblige us to define the code word c(A, B) of an 
interval [A, B[C [0, 1| as follows: 


c(A, B) = ajag--- a, —>A=0-ajaq:--a, * (the beginning of the binary 
notation of the real number A), with 
1 = [—Log.(B — A)]. 


We insist: the code word c(A, B) of an interval [A, B| is the initial segment 
of the binary notation of the left boundary A of this interval. One considers as 
much leading digits as the “information content of the interval” —Log,(B-— A) 
demands. 
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By the way: since the length B — A of an interval [A, B[C [0,1[ can actu- 
ally be considered as the probability “of falling inside” — for certain evident 
geometric experiences — the value Logs za has indeed the flavour of an in- 
formation content. 


Exercises 


Recall: binary notation of a real number A, 0 < A <1. 

Assume that the development has already been established: A = 
0-ayaza3a4:-:. Let us rediscover one by one the digits a,,a2,@3,Q4,... 
Multiply by 2: 2A = a1 - aga3a4--- (a comma-shift) 


If 2A > 1, then a; = 1, 
2A <1, then a; = 0. 
First case: pass to AG) = 2A—1=0-aga3aq4:--, 
Second case: pass to AW) = 2A =0- agaza4::-, 
And so on... 


=a 


Example Binary notation of A = 7,. 


A =0-ayjaqa3Q4°°- , 


2 
2A Sa ke OO, 
4 
aA 0, 
8 
8A ge = a3 = 0, 
16 5 
164 =— =14+— = 
6 i ay => a4 ’ 
5 
AM = 2 ~o. Eve 
ll 0- a5aga7z ) 
1 
2A@ =F <1 = a;=0, 
20 9 
4A=—s1+— => agai 
rl ame a6 ’ 
9 
AQ) = 77 — 0 aresag:: 
18 7 
PAG) ee SS ee =1 
ll a a7 ’ 
7 
AM = 77 — 0° aseacio---» 
14 
pA) ee = ag=l, 


11 11 
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AOr = SO aida 
2A{8) =F «1 = ayg=0, 
4A) =fa14e = ayo=1, 
AG) = i =0-a4jQ42013°+: = A. 
11 
The binary development of A = 4 = 0- 000I0ITIOI (period length 10). 


(1) Consider the subdivision of the interval [0,1[ by iterated dichotomy: 


00 01 10 11 


Let us encode these standard-intervals by the paths which point at them: 


1 3 
010 = ints at ee 
<<  pointsa E |. 


3.7 
110 = i t |-,=]. 
0=——+ points a E :| 
Show that the arithmetic code word c(A, B) of a standard-interval [A, B[ 
equals the binary word that points at this interval (in the tree of dichotomy 
above). 


Solution Let us show that the path a,;a2---a; which points at the interval 
[A, B[ located on level J of our tree of dichotomy is equal to the | first bits 
(after the comma) of the binary notation of A. 

Recursion on l: 

1 = 1: 0 and 1 are, respectively, the first bit of the binary notation of 
0 = 0.0000... and of } = 0.1000... 

1 + 1+ 1: consider [A, B[ on level | + 1 of our tree of dichotomy. [A, B[ 
is either the first half or the second half of an interval [A*, B*[ on level J. 

By recursion hypothesis: A* = 0- aja2---az,, and aya2---ay, points at 
[A*, B*|. If [A, B[ is to the left, then aja2---+a)0 points at [A, B] and we have 
A= A* = 0-ayag--- a0. 

If [A, B[ is to the right, then a,ag---a;1 points at [A, B[ and we have 
A = A* + 3(B* — A*) = A* + sy, ie. the binary notation of A is A = 
0- a ,a2---aj,1, as claimed. 
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(2) Find the following code words c(A, B) 
(a) ¢(3,3) 
(b) (5, 8) 
(c) (3, 4) 
(d) ¢(4,%) 


(3) Determine all intervals [A, B[ such that c(A, B) = 10101. 
(Find first the standard-interval described by 10101, then think about the 
extent you can “deform” it without changing the code word.) 

(4) Consider a memoryless source which produces N letters ao, @1,...,@N—1, 
according to the probability distribution p = (po, p1,..-,DN-—1)- 
Assume po > pi >-+: > Pn-1. 
Show that the associated Shannon code is a prefix code. 


Solution First, we have necessarily: lo < I, < --- < ly—1 (where 1; = 
[I(a;)] = [—Loggp,;| is the length of the jth code word 0 < 7 < N —1). 
Let us show that the code word c; = c(A;,Aj41) cannot be a prefix of the 
word ¢;41 = c(Aj41,A;+2) for 0 < 7 < N — 2. Otherwise we would have: 


A; =0-ajag-+-ay,* 


j 2 
Ajai =0- ayaQg:: “1, *, 
hence p; = Aj41 — Aj = 0-0---0* (with a block of at least 1; zeros after the 
comma) => p; < 274) => 1, < I(aj;), a contradiction. 
Finally, if the code word c; is a prefix of the code word cz, 7 < k, 
then c; must be necessarily a prefix of the word c;+1 (why?), and we can 
conclude. 


(5) A memoryless source produces four letters A, B, C, D, with 


pA) = 5, r(B)= 5, v(C)=p(D) = 5 

Write down the Shannon code word of BADACABA. 

Consider the source which produces the eight letters ag, a,,...,a@7, accord- 

ing to the probability distribution p = (po,p1,.-.,p7) where po = $,p1 = 

5) P2 = D3 = 7, P4 = Ps = Po = P7 = H- 

Find the associated Shannon code. 

(7) Our memoryless source produces the eight letters A, B, C, D, E, F, G, H 
according to the probability distribution p = (p(A), p(B),...,p(H)) with 


— 
aD 
Rue 


(a) Find the associated Shannon code. 
(b) Compute the average word length | of the code words. 
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Four years after Shannon’s seminal papers, the Huffman algorithm appears, 
with universal acclaim. Being of utmost mathematical simplicity, it yields 
nevertheless the best — and thus definitive — algorithmic solution of the prefix 
coding problem for memoryless discrete sources. 


Example Recall our source which produces the eight letters ao, a1,...,@7, ac- 


cording to the probability distribution p = (po, p1,..-,p7) with po = Pi = 
jo Po = Ps = 7g.P4 = Ps = De = P7 = H- 


We did encode as follows: 


aor O aqgt— 11100 
ay’ 10 as+— 11101 
a2 1100 ag += 11110 
a3 1101 a7+— 11111 


With the standard-interval coding of the preceding section in mind, where the 
code words are paths in a binary tree, one could come up with the following 
idea. 

Let us interpret the code words above as paths in a binary tree which 
admits the symbols ao, a1,...,@7 as leaves (i.e. as terminal nodes). 

We will obtain the following structure: 


ao 
0 
ay 2 1 
a2 
0 0 
1 1 
a3 
a4 : 
0 
1 
a5 
0 
1 
a6 
0 
1 
a7 


How can we generate, in general, this binary tree in function of the source 
symbols, more precisely: in function of the probability distribution p which 
describes the production of the source? 

Let us adopt the following viewpoint. 

We shall consider the given symbols as the leaves (as the terminal nodes) 
of a binary tree which has to be constructed. The code words associated with 
the symbols will be the paths towards the leaves (the symbols). 
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We note: the lesser the probability of a letter is, the longer should be 
the path towards this letter. The algorithm will have to create nodes (an- 
tecedents), conducted primarily by the rare letters; thus we shall need a nu- 
merical control by a weighting of the nodes. 

The most primitive algorithm that we can invent — based on these design- 
patterns — will actually be the best one: 


Algorithm of Huffman for the construction of a weighted binary tree: 
Every step will create a new node, antecedent for two nodes taken from a 
list of candidates. 


Start: Every source symbol is a weighted node (a candidate), the 
weight of which is its probability. 
Step: The two nodes of minimal weight (in the actual list of can- 


didates) create an antecedent whose weight will be the sum 
of the weights of its successors; it replaces them in the list of 
the candidates. 

End: There remains a single node (of weight equal to 1) in the list 
of the candidates. 


This is a recursive algorithm. Note that with every step the number of 
(couples of) nodes searching an antecedent becomes smaller and smaller. On 
the other hand, the sum over all weights remains always constant, i.e. equal 
to 1. 

Attention: “the two nodes of minimal weight” are, in general, not unique. 
You frequently have to make a choice. So, the result of Huffman’s algorithm 
is, in general, far from unique. 

Back to our example: 


1 
a0 5 
ay + 1 
a2 76 eee j , 
8 2 
a3 76 eee 
1 
44 39 i 4 
16 
Q5 35 
2 
1 
8 
a6 35 j 
16 


a7 35 
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First, it is ag and a7 which find an antecedent with weight TH (we make a 
choice at the end of the list). At the next step, we have no choice: it is a4 and 
as which have minimal weight and will thus find their antecedent of weight 
a Now, we have six nodes as candidates, four of which have weight i Once 
more, we shall choose at the end of the list, and we find this way a common 
antecedent for a4, a5, ag and az, the weight of which is = And so on... 


Exercises 


(1) Our memoryless source producing the eight letters A, B, C, D, E, F, G, H 
according to the probability distribution p = (p(A), p(B),...,p(H)) with 


p(A) = 3 W(B)=~(C)= 7% P(D) = ze, 
P(E) = p(F) = ga PG) = x, P(H) = & 


(a) Find the associated Huffman code. 
(b) Compute the average word length / of the code words. 

(2) Consider a source which produces the 12 letters ao, a1,...,@11 according 
to the probability distribution p = (po, p1,---, P11) with 


3 5 1 
pore Gt Pl = 39> - P2= 3) Fi 
P3 = P4= 39, PS =P6-—P7—Ps=— 7 PI = Pio = Pll = 35- 


(a) Compute H(p). 

(b) Find the associated Huffman code, and compare /, the average word 
length of the code words with H(p). 

(3) A memoryless source producing the three letters A, B, C with the prob- 
abilities p(A) = 3, p(B) = % and p(C) = =. 

(a) Compute the entropy of the source, find the associated Huffman code 
and the average word length for the (three) code words. 

(b) Consider now the same source, but as producer of the nine symbols 
AA, AB, AC, BA, BB, BC, CA, CB, CC, according to the product 
distribution (i.e. p(AB) = p(BA) = 3). 

Generate the associated Huffman code, and compute the average word 

length of the code words per initial symbol. Compare with (a). 


Remark The compressed bit-rate p of an encoder is defined as follows: 


average length of the code words 


average length of the source symbols’ 


It is clear that this definition makes sense only when complemented by an 
evaluation of the stream of source symbols: what is the average length of the 
source symbols — in bits per symbol? 
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(4) A binary source, which we consider as memoryless on words of length 4. 


(10 


nN 


WN 


nN 


) 


We shall adopt the hexadecimal notation (example: d = 1101). 

We observe the following probability distribution: 

p(0) = 0.40, p(4) = 0.01, p(8) = 0.01, p(c) = 0.05, 

p(1) = 0.01, p(5) = 0.03, p(9) = 0.04, p(d) = 0.01, 

p(2) = 0.01, p(6) = 0.04, p(a) = 0.03, p(e) = 0.01, 

p(3) = 0.05, p(7) = 0.01, p(b) = 0.01, p(f) = 0.28. 

Generate the associated Huffman code, and compute the compressed bit- 
rate. 

A facsimile system for transmitting line-scanned documents uses black 
runlengths and white runlengths as the source symbols. We observe the 
following probability distribution: 


p(B1) = 0.05, p(B5) = 0.01, p(W1) = 0.02, p(W5) = 0.01, 
p(B2) = 0.02, p(B6) = 0.02, p(W2) = 0.02, p(W6) = 0.01, 
p(B3) = 0.01, p(B7) = 0.01, p(W3) = 0.01, p(W7) = 0.01, 
p( BA) = 0.10, p(B8) = 0.25, p(W4) = 0.05, p(W8) = 0.40. 


As to the notation: B3 = 000, W5=11111. 

(a) Find the Huffman code for this system. 

(b) Compute the compressed bit-rate. 

A Huffman code associated with an alphabet of eight letters; we have the 

following eight code words (where two of them are masked): 

00, 10, 010, 1100, 1101, 1111, wy, wo. Find w; and wy. 

An alphabet of eight letters ao, a1, @2, @3, G4, 45, a6, 47. 

A Huffman encoder has associated the following eight code words: 

co = 00, cy = O01, co = 100, cz = 101, cg = 1100, c5 = 1101, cg = 1110, 

c7 = 1111. 

Find a probability distribution p = (po, p1,p2, 3, P4, P5, Pb, P7) Such that 

the Shannon encoder yields the same list of code words. 

A binary source which is memoryless on the eight binary triples. 

A Huffman encoder associates the eight code words 00, 01, 100, 101, 

1100, 1101, 1110, 1111. 

(a) Find a probability distribution which fits with this code. 

(b) Is it possible to choose a probability distribution which gives rise to a 
compressed bit-rate of 70%? 

Let ado be the symbol of highest probability po of an alphabet which has 

N symbols (N > 3). A Huffman encoder associates a binary code word of 

length Jo. Show the following assertions: 

(a) If po > #, then Ip = 1. 

(b) If po < §, then Ip > 2. 

The optimal questionnaire. 

You would like to participate in a TV game: you will have to find the 

profession of a person (“chosen at random”) by three yes or no questions. 

You look at the statistics: there are 16 main professions P1, P2,..., P16, 

occurring with the following frequencies: 
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p(P1) = 0.40, p(P5) = 0.05, p(P9) = 0.02, p(P13) = 0.01, 
p(P2) = 0.18, p(P6) = 0.04, p(P10) = 0.02, p(P14) = 0.01, 
p(P3) = 0.10, p(P7) = 0.03, p(P11) = 0.02, p(P15) = 0.01, 
p(P4) = 0.06, p(P8) = 0.03, p(P12) = 0.01, p(P16) = 0.01. 


(a) Find the strategy for the optimal questionnaire. 
(b) Will you have a good chance (with only three questions)? 


Huffman Coding in JPEG 


Situation JPEG treats a digital image as a sequence of blocks of 8x8 pixels. 
More precisely, a data unit will be a triple of 8 x 8 matrices. The first one 
for the pixel-values of luminance (Y), the two others for the pizel-values of 
chrominance (Cb, Cr). 


A linear invertible transformation (the 2D Discrete Cosine Transform) will 
transform each of these three matrices in a 8 x 8 matrix of the following type: 


Significant values 


Significant Values 


In lossy compression mode, an appropriate quantization procedure will 
finally set to zero most of the less significant values. 

We ultimately come up with quantized schemes (of 64 integers) of the 
following type: 


Significant 


quantized values Frequently, 


Zero quantized Values 


The value of the DC coefficient (direct current) in the left upper corner of 
the matrix will not be interesting — at least in the present context. 

The Huffman coding deals with the 63 AC coefficients (alternating cur- 
rent), the absolute values of which are — in general — sensibly smaller than 
(the absolute value of) the dominant DC coefficient. 

We shall make use of a sequential zigzag reading according to the scheme 
on the top of the next page. 
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The encoding concerns the sequence of the non-zero coefficients in the 
zigzag reading of the quantized scheme. It is clear that we also have to take 


into account the zero runlengths between the non-zero coefficients. 


DC | 1 5 6 14 | 15 | 27 | 28 
2 4 7 13 | 16 | 26 | 29 | 42 
3 8 12 | 17 | 25 | 30 | 41 | 48 
9 11 | 18 ) 24 | 31 | 40 | 44 | 53 
10 | 19 | 23 | 32 | 39 | 45 | 52 | 54 
20 | 22 | 33 | 38 | 46 | 51 | 55 | 60 
21 | 34 | 37 | 47 | 50 | 56 | 59 | 61 
35 | 36 | 48 | 49 | 57 | 58 | 62 | 63 


Zigzag ordering of the quantized coefficients. 


In order to prepare Huffman coding conveniently, we begin with a hierarchy 


of 10 categories for the non-zero coefficients: 


I =I I 
2 SS 29 a53 

3 —7, —6, —5, —4 4, 5, 6,7 

4 a1, 1 2078 8,9,...,14,15 

5 2130: ss 16 16; 17,2:., 30,31 

6 63°62 ca 90 32, 33,...,62,63 

7|  —127,—126,...,—65, —64 64, 65,..., 126, 127 

8 | —255,—254,...,-129,-128 128,129,... , 254,255 

9| —511,510,...,—257, —256 256, 257,...,510, 511 
10): 10023; = 1,029: = 513.-— 51D $12, 918,259 1,022.1, 028 


Attention There is a tacit convention concerning the encoding of all these 


integers; in category 4 for example, the code words will have four bits: 


—15 + 0000, -14+ 0001,...,-9+ 0110, —8 + 0111, 8+ 1000, 94 


1001,...,14+ 1110, 15 


1111. 
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We observe that a non-zero coefficient occurring in the sequential reading 
of a quantized scheme can be characterized by three parameters: 


(1) The number of zeros which separate it from its non-zero predecessor. 
(2) Its category. 
(3) Its number within the category. 


Example Consider the sequence 0800—2040001... 
This means for 


Runlength/category| Value within the cat. 
8 1/4 1000 
=o 2/2 01 
4 1/3 100 
1 3/1 1 


In order to be able to encode the sequential reading of the quantized 
coefficients, we need only a coding table for the symbols of the type run- 
length/category. 

We shall give the table for the luminance AC coefficients. 

The table has been developed by JPEG (Joint Photographic Experts 
Group) from the average statistics of a large set of images with 8 bit pre- 
cision. It was not meant to be a default table, but actually it is. 


Remark On two particular symbols. 


(1) (EOB) = end of block indicates the end of the non-zero coefficients in the 
sequence of the 63 AC coefficients to be encoded. The code word for this 
happy event will be 1010. 

(2) (ZRL) = zero run list indicates the outcome of the integer 0 preceded by 
a block of 15 zeros.? 


2 Attention, our zeros are zeros as integers — and not as bits! 
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Runlength/cat.|Code word Runlength/cat.|Code word 

0/0 (EOB) |1010 3/9 1111111110010100 
0/1 00 3/a 1111111110010101 
0/2 O1 4/1 111011 
0/3 100 4/2 1111111000 
0/4 1011 4/3 1111111110010110 
0/5 11010 4/4 1111111110010111 
0/6 1111000 4/5 1111111110011000 
0/7 11111000 4/6 1111111110011001 
0/8 1111110110 4/7 1111111110011010 
0/9 1111111110000010 4/8 1111111110011011 
0/a 1111111110000011 4/9 1111111110011100 
1/1 1100 4/a 1111111110011101 
1/2 11011 5/1 1111010 
1/3 1111001 5/2 11111110111 
1/4 111110110 5/3 1111111110011110 
1/5 11111110110 5/4 1111111110011111 
1/6 1111111110000100 5/5 1111111110100000 
1/7 1111111110000101 5/6 1111111110100001 
1/8 1111111110000110 5/7 1111111110100010 
1/9 1111111110000111 5/8 1111111110100011 
1/a 1111111110001000 5/9 1111111110100100 
2/1 11100 5/a 1111111110100101 
2/2 11111001 6/1 1111011 
2/3 1111110111 6/2 111111110110 
2/4 111111110100 6/3 1111111110100110 
2/5 1111111110001001 6/4 1111111110100111 
2/6 1111111110001010 6/5 1111111110101000 
2/7 1111111110001011 6/6 1111111110101001 
2/8 1111111110001100 6/7 1111111110101010 
2/9 1111111110001101 6/8 1111111110101011 
2/a 1111111110001110 6/9 1111111110101100 
3/1 111010 6/a 1111111110101101 
3/2 111110111 7/1 11111010 
3/3 111111110101 7/2 111111110111 
3/4 1111111110001111 7/3 1111111110101110 
3/5 1111111110010000 7/4 1111111110101111 
3/6 1111111110010001 7/5 1111111110110000 
3/7 1111111110010010 7/6 1111111110110001 
3/8 1111111110010011 7/7 1111111110110010 
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Runlength/cat.|Code word Runlength/cat.|Code word 

7/8 1111111110110011 b/7 1111111111010101 
7/9 1111111110110100 b/8 1111111111010110 
7/a 1111111110110101 b/9 1111111111010111 
8/1 111111000 b/a 1111111111011000 
8/2 111111111000000 c/1 1111111010 

8/3 1111111110110110 c/2 1111111111011001 
8/4 1111111110110111 c/3 1111111111011010 
8/5 1111111110111000 c/4 1111111111011011 
8/6 1111111110111001 c/5 1111111111011100 
8/7 1111111110111010 c/6 1111111111011101 
8/8 1111111110111011 c/7 1111111111011110 
8/9 1111111110111100 c/8 1111111111011111 
8/a 1111111110111101 c/9 1111111111100000 
9/1 111111001 c/a 1111111111100001 
9/2 1111111110111110 d/1 11111111000 

9/3 1111111110111111 d/2 1111111111100010 
9/4 1111111111000000 d/3 1111111111100011 
9/5 1111111111000001 d/4 1111111111100100 
9/6 1111111111000010 d/5 1111111111100101 
9/7 1111111111000011 d/6 1111111111100110 
9/8 1111111111000100 d/7 1111111111100111 
9/9 1111111111000101 d/8 1111111111101000 
9/a 1111111111000110 d/9 1111111111101001 
a/1 111111010 d/a 1111111111101010 
a/2 1111111111000111 e/1 1111111111101011 
a/3 1111111111001000 e/2 1111111111101100 
a/4 1111111111001001 e/3 1111111111101101 
a/5 1111111111001010 e/4 1111111111101110 
a/6 1111111111001011 e/5 1111111111101111 
a/7 1111111111001100 e/6 1111111111110000 
a/8 1111111111001101 e/7 1111111111110001 
a/9 1111111111001110 e/8 1111111111110010 
a/a 1111111111001111 e/9 1111111111110011 
b/1 1111111001 e/a 1111111111110100 
b/2 1111111111010000}, f/0 (ZRL) {11111111001 

b/3 1111111111010001 f/1 1111111111110101 
b/4 1111111111010010 f/2 1111111111110110 
b/5 1111111111010011 f/3 1111111111110111 
b/6 1111111111010100 f/4 1111111111111000 
f/5 1111111111111001 f/8 1111111111111100 
f/6 1111111111111010 f/9 1111111111111101 
f/7 1111111111111011 f/a 1111111111111110 
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Example Consider a luminance data unit in gradation, its 2D DCT image, 
then the quantized scheme? 


30 30 30 30 30 30 6 6306 «630 1,080 0 0 0 0 0 0 0 
60 60 60 60 60 60 60 60 —546.6 0 0 0 0 0 0 0 
90 90 90 90 90 90 90 90 0 000 0 0 0 0 
120 120 120 120 120 120 120 120 —57.1 0 0 0 0 0 0 0 
—— ad 

150 150 150 150 150 150 150 150 0 000 00 0 0 
180 180 180 180 180 180 180 180 —17 000 0 0 0 0 
210 210 210 210 210 210 210 210 0 00 0 00 0 0 
240 240 240 240 240 240 240 240 —4.3 000 0 0 0 0 
bee ae 68 is the round of 4?" 

-46 0 0 0 0 0 0 0 Pe 

0 000000 0 —46 is the round of 53> 

4 00 00 0 0 0 <= —4 is the round of sit 

0 0 0 0 0 0 0 +0 = 

2a Ag 40 OO ee —1 is the round of => 

ODS Re ae ed 0 is the round of =32 

0 0 0 0 0 0 0 0 


The sequence of the 63 AC coefficients: 


0-46000000-40000000000-10000000000000000 
0DDDDDDDDDDDDDDDDDDDDVDVNVDONNND. 


We have to encode the following symbols: (1/6)[n°17](6/3)[n°3](a/1) [n°0} 
(EOB). 
The code word: 1111111110000100 010001 1111111110100110 011 
111111010 0 1010.4 
As to the compressed bit-rate, we have associated with 64 8-bit bytes (the 
55 


8 x 8 luminance values) a code word of length 55. Hence: p = 35 = 0.11. 


It is evident that the code word permits the reconstruction of the ma- 
trix of the quantized values (the quantized DC coefficient has its particular 
treatment...). Dequantization simply means multiplication with the divisor- 
scheme that has been used for quantization. In our particular case, we get the 
following dequantized matrix: 


3 Quantization is done by means of a fixed scheme of 8 x 8 divisors, followed by a 
round to the next integer. 

4 Attention: we have got a prefix code — the separating blanks in our notation are 
redundant and exist only for the convenience of the reader... 
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68-16 0 0 0 0 0 0 0 1,088 0 0 0 0 0 0 0O 
—46-12 0 0 0 0 0 0 0 —552 0 0 0 0 0 0 O 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
-4-14 0 0 0 0 0 0 0O _ —-56 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 a 0 0 0 0 0 0 0 0 
-1:24 0 0 0 0 0 0 0O -24 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0.72 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 


This is the moment where the loss of information, due to quantization, 
becomes apparent. The decompression will be finished after transforming the 
matrix of dequantized values back, and making the necessary rounds (do not 
forget: we did start with 8-bit bytes). We obtain: 


30 30 30 30 30 30 30 30 
61 61 61 61 61 61 61 61 
91 91 91 91 91 91 91 91 
119 119 119 119 119 119 119 119 
153) 153) 153) 158) 153) 1538) 153 158 
181 181 181 181 181 181 181 = 181 
211) 211 211 211 211 211 211 211 
242 242 242 242 242 242 242 242 


Exercises 


(1) The Huffman coding described above has a two-level structure: there are 
code words for the type of the integer to be encoded, and there are the 
code words for its numerical value. Writing down the total Huffman tree 
for this coding, what will be the number of leaves (of code words)? 

(2) We shall treat five 8 x 8 matrices of luminance values, together with their 
quantized 2D DCT matrices. Establish in every case the associated Huff- 
man coding, and compute the compressed bit-rate. 


159 152 142 134 133 140 149 155 64000000 0 
176 170 162 156 157 163 171 177 00400000 
132 129 123 120 121 126 132 136 170000000 
(a) 72 71 69 68 69 70 72 74 ome, 00000000 
69 70 72 73 73 71 69 67 -90000000 
123 126 131 134 133 129 123 119 00000000 
157 163 171 177 176 170 162 156 00000000 
132 139 149 157 158 151 142 135 00000000 
83 89 91 84 73 68 75 83 50 0 0 -3 0000 
96 98 96 86 78 82 99 114 -90 2 00000 
82 85 84 75 67 69 83 97 010 00000 
(b) 88 99 108 107 99 94 97 104 — 100 00000 
88 101 115 118 112 108 111 116 0 0-1 0 0000 
95 103 110 107 99 97 105 114 000 00000 
122 130 136 131 120 114 117 124 -10 0 00000 
96 111 127 131 121 109 105 105 000 00000 
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25 32 41 46 47 #48 51 53 41 -—5 1 -3 0000 
45 57 75 90 94 89 80 73 -15 2 -4 10000 
26 34 43 48 46 41 37 34 0 -3 0 1 0000 
(c) 82 89 92 84 72 69 78 90 aan 1 O 0 -10000 
87 101 109 98 80 79 102 126 0 O 1 00000 
61 78 91 84 67 65 87 110 0 0 0 00000 
112 122 131 129 123 126 141 156 -3 0 0 00000 
94 88 80 77 85 106 131 149 0 0 0 00000 
133 139 89 198 114 91 151 114 63 0 0 0 1 0 -1 0 
126 127 95 207 136 110 148 124 0 1 0-10 1 =0 -1 
134 128 186 73 32 158 93 127 2 0-2 0 1 0 -1 0 
(d) 104 128 75 40 246 129 143 141 fal 0 1 0-10 1 =0 -1 
141 143 129 246 40 75 128 104 0 0-10 1 0-1 0 
127 93 158 32 73 186 128 134 0-10 1 0-10 1 
124 148 110 136 207 95 127 126 -1 0 1 0-10 1 #0 
114 151 91 114 198 89 139 133 0 1 0-10 1 #0 -1 
65 61 68 59 69 60 67 63 320000000 
61 72 52 78 50 76 56 67 00000000 
68 52 81 44 84 47 76 60 00000000 
fee ere eg suo Oe 
69 50 84 40 88 44 78 59 00000000 
60 76 47 84 44 81 52 68 00000000 
67 56 76 50 78 52 72 61 00000000 
63 67 60 69 59 68 61 65 00000001 


We observe a stubborn propagation of non-zero values in the quantized 
scheme (d). This comes from a (pictorially) perverse distribution of 
the luminance values in the initial scheme. Note that in general the 
zigzag ordering has the property that the probability of coefficients 
being zero is an approximately monotonic increasing function of the 
index. 


Huffman Coding is Optimal 


Situation A memoryless source, producing N letters ag,a1,...,@N—1 @c- 
cording to the fixed probability distribution p = (po, pi,---,;PN-—1)- 


Now consider all associated binary prefix codes. 
Such a binary prefix code Cis optimal :<==> 
The average word length 1 = polo + pili + -:- + pn—iln—1 of its words is 
minimal. (Note that 1 = I(Io,l1,...,l—1) is a function of the lengths of the N 
code words associated with ag, a,,...,@n—1; the probabilities po, p1,...,pN—1 
are the constants of the problem.) 

Our goal: we shall show that the Huffman algorithm necessarily produces 
an optimal binary prefix code. 
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But first several characteristic properties of optimal codes: 


Proposition Let C be an optimal binary prefix code, associated with p = 
(po. P1; tee ,PN-1)- 


Then we necessarily have: 


BY 


pj > Pe 1 < Ih. 

The code will have an even number of words of maximal length. 

3. Whenever several code words have the same length, two of them will be 
equal except for the last bit. 


ve 


Proof 1. Assume p; > pp andl; > ly; then 


(pj — Pe)(lj — lk) > 9, 
i.e. 
ply + pele > pile + prl;. 
This shows: if we exchange the code words of a; and of ax, then we get a 
better code. 

2. Otherwise there would exist a code word of maximal length without a 
partner which differs only in the last bit. We can suppress its last bit, 
and will obtain a word which is not a code word (prefix property!). This 
permits us to change to a better code. 

3. Consider all code words of length | (arbitrary, but fixed). Assume that all 
these words remain distinct when skipping everywhere the last bit. Due 
to the prefix property, we would thus get a better code. 


The Huffman codes are optimal: this is an immediate consequence of the 
following proposition. 


Proposition Consider a source S of N states, controlled by the probability 
distribution p = (po, P1,---,;PN—1)- 

Replace the two symbols a;, and aj, of smallest probabilities by a single 
symbol ay;, j.) with probability p(j,,j.) = Pj1 +Pj2- Let S’ be the source of N—1 
states we get this way. 

Let C’ be an optimal binary prefix code for S’, and let x be the code word 
of A(j1,52)- 


aj, -—> x0, 


J. 
J2 


Let C be the ensuing binary prefix code for S: 
Then C is optimal for S. 


Proof The lengths of the code words (L for C’, | for C): 
b= ee +1, if 7 = ji, je, 
I Lj, else. 
One gets for the average lengths (L for C’, 1 for C): 
LM iirsig Bila + Pili + Piobjn = yj djz jn PILI + Plir.i2) LE Gade) + Pir + 


Pje =L+ py, + Pj: 
But p;, + pj, is a constant for our optimality arguments (we make the 
word lengths vary), and it is the constant of the smallest possible difference 


between / and L (due to the choice of p;, and p;,!). Thus, L minimal => | 
minimal. 


> xl. 
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Corollary The Huffman algorithm produces optimal binary prefix codes. 

In particular, the average word length | of the code words is constant for 
all Huffman codes associated with a fixed probability distribution p (note that 
you will be frequently obliged to make choices when constructing Huffman 
trees). 


Observation For every Huffman code associated with a fixed probability dis- 


tribution p we have the estimation H(p) < | < H(p)+ 1 (this is true for 
Shannon coding, hence a forteriori for Huffman coding). 


Exercises 


(1) Does there exist a binary prefix code which consists of three words of 

length 3, of five words of length 4 and of nine words of length 5? 

Does there exist a Huffman code which consists of three words of length 

3, of five words of length 4 and of nine words of length 5? 

Show: every Huffman code is a Shannon code. 

More precisely: let C be a Huffman code (given by its binary tree); then 

there exists a probability distribution p such that the set of code words 

of C is the associated Shannon code. 

Let {A,B,C,D} be an alphabet of four letters, with p(A) > p(B) > 

p(C) = p(D). 

(a) Find all associated Huffman codes. 

(b) Give an example of a Shannon code (in choosing appropriate proba- 
bilities) which is not a Huffman code. 

(4) Is an optimal binary prefix code necessarily a Huffman code? 


— 
i) 
Sore 


— 
wo 
Ww 


Approximation of the Entropy via Block Encoding 


Consider a memoryless source, producing the N letters ao,a1,...,@N—1, ac- 
cording to the probability distribution p = (po, pi,.-.,;PN—1)- 

Let us change our outlook: 

We shall take the words of length n (n > 1) as our new production units: 

X = 4j,Gj. +++ a;, Will be the notation for these “big letters”. 

The product probability distribution: 


p [ [21.2% Pin O<91,525--.5n<N-1 


e (there are N” words of length n on an alphabet of N elements). 


Recall 
H(p™) =n- H(p) 


(this was a previous exercise). 
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Proposition A memoryless source, producing the N letters ag,a1,.--,@N—1; 
according to the probability distribution p = (po, P1,---;DPN-—1)- 

Let us pass to an encoding of blocks in n letters. 

Let C be an associated Huffman code, and let 1; be the average word length 
of the code words per initial symbol. 

Then we have: 


H(p) <1. < H(p) + ~. 


Proof Let l, = >> p(x)l(x) be the average length of the code words of C. We 
have the following estimation: 


H(p™) <In < H(p™) +1. 


But H(p™) =n- H(p), In = n-;; whence the final result. 


Exercises 


(1) Consider a binary source which produces one bit per unit of time (for 
example, every Us). We know: po = 3, p= i: 

Suppose that our bitstream has to pass through a channel which accepts 
only 0.82 bits per unit of time. Construct an adapter by means of Huffman 
block encoding. 

(N.B.: H(p) = 0.81). 

A memoryless source, producing the N letters ag, a1,...,@n—1, according 
to the probability distribution p = (po, P1,---;PN—1)- 

Let C = C, be an associated binary prefix code, and let J be the average 
word length of its code words. 

For n > 1 let us encode as follows: 


— 
i) 
YS 


X = Oj, jy °** 5, +> C(X) = C(a;,)c(aj,) +++ c(a;,,). 


Let Cy, be the ensuing code; n > 1. 

(a) Show that C,, is a binary prefix code associated with p(, n > 1. 

(b) Show that we have, for each C,,: 1; = I. 

(c) Give a necessary and sufficient condition for all the C,, to be optimal 
(with respect to p(”). 


1.1.3 Arithmetic Coding 


The arithmetic coding is a dynamic version (presented by a recursive algo- 
rithm) of Shannon block encoding (with continually increasing block lengths). 
Actually, all of these block codes will be visited very shortly: for every block 
length n > 1, we shall encode only a single word of length n: the initial 
segment of length n of a given stream of source symbols. 

Arithmetic coding is much easier to explain than to understand. That is 
why we shall adopt — at least at the beginning — a very pedantic viewpoint. 
Towards the end, we shall be concerned with more practical aspects of arith- 
metic coding. 
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The Elias Encoder 


The Elias encoder was initially meant to be a purely academic construction. 
Its first (and rather discreet) presentation dates from 1968. It was between 
1976 and 1979 (Pasco, Jones, Rubin) that arithmetic coding began to be 
considered as a practically interesting method for lossless compression. 


The situation A memoryless source, producing the N letters ao, a1,..., 
an-—1, according to the probability distribution p = (po, pi,...,;DN-—1)- 


We shall always suppose pp > py > ++: > pn-1. 

The arithmetic encoder will associate with a stream of source sym- 
bols aj,aj;,+-+@;,°*: (which could be theoretically unlimited), a bitstream 
Q1.0203 ++: Q,--+ (which would then also be unlimited). 

But let us stop after n encoding steps: 

The code word a ,a2a3--- a; of | bits associated with the n first source 
symbols aj,a;,---a;, will be the code word c(a;,a;,---a@;,) of a Shannon 
block encoding formally adapted to recursiveness according to the device: 
“every step yields a tree-antecedent to the next step”. 

We shall, in particular, inherit from the preceding section: if the actual 
production of the source is statistically correct, then | will be the average 
length of the code words for a Shannon block encoding, and consequently 


l 1 
H(p)<-<H —. 

(p) <— < Alp) + — 
In this sense, the arithmetic encoder will work optimally, i.e. very close to the 
entropy of the source. The number of bits produced by the arithmetic encoder, 
counted per source symbol, will be asymptotically equal to the entropy of the 
source. As a first initiation in arithmetic coding, let us consider a “superfluous” 


version. 


Example A (memoryless) source, producing the four letters a, b, c, d according 


to the probability distribution given by p(a) = 3, p(b) = §, p(c) = p(d) = =. 


The Shannon code: 


atr—>0 0 = 0.0000... 
br 10 + = 0.10000... 
cH 110 3 = 0.110000... 


dt— 111 £ = 0.111000... 


Now, look at the source word badacaba. Its code word is 10011101100100. 
Let us write down this concatenation in “temporal progression” : 
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b 10 

ba 100 

bad 100111 

bada 1001110 

badac 1001110110 
badaca 10011101100 
badacab 1001110110010 
badacaba 10011101100100 


Recall our binary tree of all standard-intervals obtained by iterated 
dichotomy. 


10 is the code word of the interval 3, 3[, 

100 is the code word of the interval 5, 3, 

100111 is the code word of the interval [3, 3 ls 
1001110 is the code word of the interval [3, [, 
1001110110 is the code word of the interval Be fa [. 
10011101100 is the code word of the interval [ $18, 262 E 
1001110110010 is the code word of the interval 2.521, in F 
10011101100100 is the code word of the interval 2.52, ise F 


We have obtained a chain of eight intervals: [4. 3 [ is the interval of b for 
the Shannon partition of the interval [0, 1]. 


The interval [5 al is the interval of a for the Shannon partition of the 


278 
interval [5 3 FE 
a b Cc d 
e e e e e 
1 ei aay | 3S 
2 8 16 32 4 


The interval [#, 3[ is the interval of d for the Shannon partition of the 


: 1 5). 
interval [5 3 [: 
a b c d 
e e e e e 
1 9 19 39 5 
2 16 32 64 8 
and so on... 


Let us insist: 

b points at its interval [5 31. 

ba points at the interval of a within the interval of b. 

bad points at the interval of d within the interval of a within the interval 


of b, and so on... 


The recursive algorithm for the construction of the chain of intervals cor- 
responding to the successive arrivals of the source symbols: 
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Start: Ao =0 Bo =1, 

Step: The initial segment s152---s,, of the source stream points at 
[Am, Bm|- 

Compute three division points D,, Dz and D3 for the interval [Am, Bm|: 

D, = An Tr p(a)(Bm = Ava) 

D2 = Am + (p(a) + 2(0))(Bm — Am) 

D3 = Am + (pa) + p(b) + P(c))(Bm — Am). 

Let Sm41 be the (m+ 1)st source symbol. 


a Anas _ Aas Bn4i = Di, 

—_ b Am+1 = Dy, Bys41 = Dao, 

as sic a Cc oh Am-+1 — Do, Bm+1 > Ds, 
d Am+1 = Ds, Bms41 = Bm. 


End: m = n. The code word of 8152---+ 8, is the word c(A,, B,) =the code 
word of the interval [A,, Bp]. 


Note that the least upper bounds B, of our intervals are not very im- 
portant (and are often completely neglected): the code word is a prefix of 
the binary notation of A, and its length | is determined by the information 
content I(s152--+S,). 

Let us sum up our observations: 

The simple Shannon coding c(badacaba) = c(b)c(a)c(d)c(a)c(c)c(a)c(b)c(a) 
becomes considerably more complicated from our viewpoint of “dynamic block 
coding” and by the arithmetic generating the correct strings of intervals. Our 
first impression is that there is no advantage in adopting this new vision of 
computing code words. 

But now let us change the data. 

Our source will still produce the four letters a, b, c, d, but now according to 
a new probability distribution given by p(a) = $, p(b) = 3, p(c) = p(d) = x. 

We compute: I(a) = 0.42, I(b) = 3, I(c) = I(d) = 4. 

The Shannon code: 


a b c d 
e e e e e 
3 7 15 
0 4 8 6 
ar 0 
b+ 110 
cr 1110 
d+— 1111 


Let us choose a source word in conformity with the statistics: daaabaaa- 
caaabaaa. 

The associated Shannon code word is 11110001100001110000110000 and 
has 26 bits. 
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But if we adopt an arithmetic encoding, making use of the recursive al- 
gorithm explained earlier, we will end with a code word of | bits, where | = 
[I(daaabaaacaaabaaa)| = [122 (a)+22(b)+1(c)+1(d)| = [5.04+648] = 20. 

So, the arithmetic code word of daaabaaacaaabaaa is shorter than the (con- 
catenated) Shannon code word. This is a general fact: whenever the probabil- 
ities are not powers of T arithmetic coding is better than any block coding 
(of fixed block length). 


Exercises 


(1) The memoryless source which produces the four letters a, b, c, d, according 
to the probability distribution given by p(a) = 3, p(b) = $, p(c) = p(d) = 
Conipiite the arithmetic code word of daaabaaacaaabaaa (thus completing 
the example above). 

(2) A memoryless binary source such that po = 3,p1 = 4. 
Compute the arithmetic code word of 00101000. 

(3) A memoryless source producing the three letters a,b,c according to the 
probability distribution given by p(a) = $, p(b) = p(c) = §. 
Find the arithmetic code word of aabaacaa. 

(4) Write down the general version of the recursive algorithm for arith- 
metic coding. Recall: we have to do with a memoryless source produc- 
ing N letters ag,a,,...,@N—1, according to the probability distribution 
P = (Po, P1,---,PN-1), and po > pi > ++: > pw-1 > 0. 

(5) The situation as in exercise (4). Suppose that all probabilities are powers 
of $:p) =2-4,0<j<N-1. 

Show that in this case the arithmetic code word of a source word 8182 --+ 8», 
is equal to the Shannon code word (obtained by simple concatenation of 
the code words for 81, 82,...,n). 

(6) A memoryless source, producing the N letters ao, a1,...,a@N_—1 according 
to the (“decreasing”) probability distribution p = (po, pi,..-,;PN-—1)- 

Let s1 and sz be two source words such that c(s1) = c(s2) (they have the 
same arithmetic code word). 
Show that either s; is a prefix of sg or So is a prefix of s1. 


Solution s; points at [Am, Bm|. 

S29 points at [C;,, Dy. 

Our hypothesis: A,, = 0-ajaq---aj*, C, = 0-ayag---a;*, where | = 
| —Log2(Bm — Am)| = [—Logg(Dn — Cn)]. 

This means: 

OCB A ee OT 

7 ge ed Br Ge es I ae 

Suppose m <n. 
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We have only to show that [C,,, Dn[C [Am, Bm[ (this inclusion implies that 
s1 is a prefix of s2). 

Otherwise, we would have [C;,, Dn[N[ Am, Bm|[= 9 

Hence, |C;, — Am| > 27!, i.e. the binary notations of A», and of C,, would 
already differ (at least) at the Ith position after the comma. But this is a 


contradiction to our hypothesis above. 


The Arithmetic Decoder 


The arithmetic decoder will reconstruct, step by step, the source stream from 
the code stream. It has to discern the encoding decisions by the means of the 
information which arrive with the code stream. Every decision of the encoder 
is dedicated to the identification of a source symbol — a process which has to 
be imitated by the decoder. 

This is the occasion where our modelling in terms of (chains of) intervals 
will be helpful. 

First, consider the situation from a static viewpoint. The decoder can take 
into account the complete code stream a 1Q2:--Q@rz. 

This means that we face the code word c(Ay, By) of an interval 

Ayn = 0-a102+++ap* (* = masked part), 

L = [—Log,(Bn — An)]. 

We search: the source stream $152---S,, which points at this interval 
(in the Shannon partition tree whose structure comes from the lexicographic 
ordering of the pointers). 

Let us look at the following. 


Example A (memoryless) source producing the three letters a, b, c according 
to the probability distribution p given by p(a) = 3, p(b) = p(c) = §. 

The code word: 100110111. 

Our terminal interval [A,,, B,| is described by 

Ar, = 0.100110111«, 

gy < Bn —An < ge- 

The decoding will consist of keeping track of the encoder’s decisions. 

The logical main argument for decoding is the following: the hierarchy of 
the Shannon partition tree accepts only chains of intervals or empty intersec- 
tions. 

This means: whenever a single point of such an interval is to the left or 
to the right of a division point, then the entire interval will be to the left or 
to the right of this point (recall the solution of exercise (6) at the end of the 
preceding section). 

Let us begin with the decoding. 

First step: 

Compute D,; = = = 0.11, 

D2 = 4£=0.111, 
Ay < Dy => [An, Bn[C [0, Di[ = the interval of a. 


Col NR] Co 
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=> $, = 4. 
Thus, Ay = 0, By = 3, 
Second step: 
Compute D,; = 2 = 0.1001, 
Dz = a = 0.10101, 
Dy < An < Dz => [An, Bn[C [D1, Do[ = the interval of ab. 
= > $189 = ab. 
Thus, Ag = 2 Bo = 21 


RO 


16° 32° 
Third step: 
Compute D; = 35 = 0.1010001, 
D2 = 3% = 0.1011101, 


An < Dy => [An, Bn[C [A2, Dil = the interval of aba. 
=> 818283 = aba. 
Thus, Ag = 3. Bz = ss. 
Fourth step: 
Compute D, = 375 = 0.100111011, 
Dg = yo5q = 9-1001111111, 
Ay < Dy => [An, Bn[C [As, Di[ = the interval of abaa. 
= > $1825354 = abaa. 
Thus, Ay = 2, By = an. 
Fifth step: 
Compute D, = #223 = 0.10011010001, 
Dz = 2188 = 0.100110111101, 
An and Dz have the same binary notation — until the masked part of Ay. 


BR 


Question How shall we continue? 
Answer A, = D2 and s5 =c (i.e. [An, Bn[C [Do, Bal). 


Justification If we suppose that A, = Dz and s5 = c, then we have found a 
source word $ = 8182538485 = abaac with the code word 10011011 (note that 
I(abaac) = 3 x 0.42+3+3 = 7.26). 


Suppose, furthermore, that only the letter a was produced in the sequel (5088 


remains then the left end point As = Ag = Az, etc. of the code interval), so we 
will have $1 825384858687 = abaacaa with I(abaacaa) = 5 x 0.42+3+4+3 =8.1. 
This source word will produce our code word, but also 

8182835485 568753 = abaacaaa (I(s)--- 8g) = 8.52), 

8182835485 56878889 = abaacaaaa (I(s1---+ 89) = 8.94). 

We obtain this way three source words 

§1825354555657, 

$152535455565758, 

518253545556575859, 


which produce the same code word 100110111. 


2,493 9 


Back to the question: why An = 759g! 
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The answer: Under this hypothesis, we obtain a coherent and logical decod- 
ing. But, according to exercise (6) of the preceding section, every string of 
source symbols with the code word 100110111 must be a prefix of abaacaaaa. 


Attention The argument concerning the end of our decoding algorithm de- 
pends on a complete message: the string of the received code bits has to be the 
code word of an interval. 


Now consider an incomplete message. 

In the situation of our example, assume that the first three bits of the 
code stream are 101. This is not a code word of an interval in our partition 
tree associated with the given probability distribution. Let us try to determine 
altogether the first two source symbols. 101 is the prefix of the binary notation 
of the greatest lower bound A of an interval [A, B[ coming up at some step of 
the encoding: 


A=0.101*. 


We note: A < 3 = D, = the inferior division point of the first step of the 
encoder, 


=> 5s, =a. 


The division points of the second step (inside the interval [A;, By [= [0, ?[): 

D, = % = 0.1001, Dz = 35 = 0.10101, 

A> D, = s9=bor s2=c. 

The case 5182 = ac gives Ag = 0.10101, and this is ok. 

The case 5,52 = ab needs a closer inspection of the possible continuations. 

And indeed: the code word of 815983 = abc is 1010010, since the second 
division point D2 inside [A2, Ba/= [3 ral is Dg = ie = 0.10100101. 

We see: our rules about the decoding at the end of the code stream can 
only be applied in a terminal situation. We have to face the code word of an 


interval. 


Exercises 


(1) Write down the decoding algorithm in general, i.e. for a memoryless 
source, producing the N letters ao, a1,...,a@n—1 according to the proba- 
bility distribution p = (po, p1,---,PN—1) with po > pi > +--+: > pn-1 > 0. 

(2) A (memoryless) binary source p = (po, p1) with po = 3. p= i. Decode 
1101010. 

(3) The binary source of exercise (2). The decoder receives a code stream 
beginning with 1001. Find the first three bits of the source stream. 

(4) A memoryless source which produces the four letters a, b, c, d according 
to the probability distribution p given by p(a) = 3, p(b) = 3, p(c) = 
p(d) = 3. Decode 11101111000101010000. 

(5) A (binary) arithmetic encoder produces zeros and ones; hence, it is nothing 
but a binary source. What then is its entropy? 
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It is clear that the spirit of arithmetic coding demands a continual bitstream 
output of the encoder. On the other hand, the constraints of a bounded arith- 
metic call for perpetual renormalization in order to avoid an asphyxiation in 
more and more complex computations. So, we need a practicable version of 
the Elias encoder and decoder, i.e. a sort of prototype for effective implemen- 
tation. 

Some (almost) trivial observations 

Let [A, B[ be a (small) interval taking place in our computations. We are 
interested in the division point 

D=A+p-(B-—A) (p=acumulative probability). 

If the difference B — A is small, then we will have (generically — there are 
the well-known exceptions) 


A =adr 


B=a+s, 


where a describes the identical prefix of the binary notation® of A and of B. 
Compute d=r+p-(s—r) (subdivision of the remainder parts ...). 
Then D=a+d. 


On the other hand (stability of our computations under “arithmetic 
zoom” ): consider, for m > 1, 

R=2"r, 

S=2"s. 

Then R+p-(S— R)=2™(r+p-(s—r)). 


Consequence Suppose that the binary notations of A and of B are identical 
up to a certain position t: 


A=Q1-++Q4Q441°°°, 


B=a,:- ary. --: 


Then the computation of D = A+ p-(B-—A) is equivalent to the computation 
of T=R+p-(S—R) 
with R=0-atsi-:: 
S08 Boag 
More precisely, if T = 0-7,72---, then D=0-ay---a47]T2°°° 


This allows a renormalizable arithmetic: our computations remain in the 
higher levels of appropriate partition trees of simple intervals inside [0, 1[. 

The criterion for sending off a string of code bits will come from a simple 
syntactical comparison. 

Let [An, By[ be the interval of the source word 8182--+ Sn. 

Write A, = 0-ayag-+-Q7ar41-°° , 


Bn = 0+ G1 B2--+PBig1---. 


® We shall never consider “false” infinite notations with period “1”. 
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Tf oy = By ag = Bo-++- a4 = , then our encoder will produce (send off) 
the string of code bits ajaz2--- az and will continue the encoding with 

APE Qucires., 

Br =0-+ Bryi-+:- 


Example The (memoryless) source which produces the three letters a, b,c ac- 
cording to the probability distribution p given by p(a) = 3, p(b) = p(c) = . 
Consider the source word abaacaaaa (an old friend — look at our decoding 
example in the last section!). 
First step of the encoder: 


D, = 3=0.11, 

D,=%=0111, 

S1 a Ay 0 0.000..., 
By =3%=0.11. 


No common prefix can be discarded. 
Second step of the encoder: 
D, = £ =0.1001, 


16 
D2 = 34 = 0.10101, 
82 =b => Ap = 2 =0.1001, 
By = 35 = 0.10101. 


We send off: aya = 10. 
We renormalise: AS = 0.01 = i, 


By = 0101 = 3. 
Third step of the encoder: 
D, =}4+2-3=# =0.10001, 
Dg=4+%-3 = # =0.100101, 


s3 =a A3 = 4 = 0.01, 

Bz = $5 = 0.10001. 
No common prefix can be discarded. 
Fourth step of the encoder: 


—~1,3,9 _ 59 _ 
D, =14+3.8 = —o.011011, 


4' 4 
1 7 9. AQT —. 
Do=14+2.8=27 -ooui, 


s4=4 Ay = 4 = 0.01, 
Byz= bo = 0.0111011. 
We send off: aga4 = 01. 
We renormalise: A, = 0, 
Bi, =0.11011 = 3%. 
Fifth step of the encoder: 
Dy aoe 2e— 3) 04010001; 


4 32 128 
7, 27 _ 189 _ 
Da = § 3 = 3g = 010111101, 
85 =c—> As = #82 = 0.10111101 
Bs = 2% = 0.11011. 


We send off: as = 1. 
We renormalise: AS = 0.0111101 = a, 
Bg = 0.1011 = #. 
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Sixth step of the encoder: 


Dy is 3 : i= _ 3 = = 0. 101000101, 
56 =a Ag = As = 0.0111101, 


Be = D, = 0.101000101. 
No common prefix can be discarded. 
Seventh step of the encoder: 
Di = $e +2- 3 = Shs = 0-10011000011, 
87 =a A, = Ag = 0.011110, 
Bz = D, = 0.10011000011. 
No common prefix can be discarded. 
Eighth step of the encoder: 
Dy = Og + 3° voes = Sy93 = 0-1001000011001, 
$3 =a Ag = A7 = 0.0111101, 
Bg = D, = 0.1001000011001. 
No common prefix can be discarded. 
Ninth step of the encoder: 
D, = Oy + 3+ ses = eres = 0-100010110001011, 
$89 =a Ag = Ag = 0.0111101, 
Bg = D, = 0.100010110001011. 


No common prefix can be discarded. 


Exercises 


(1) Continue the previous example: find the shortest source word s152:-- 
$9819°*: such that the encoder will effectively send off (after the con- 
venient syntactical tests) ag@7agag = 0111. 

(2) True or false: if s;s2---s, is the beginning of the source stream, then its 
code word c(s182---,) is the beginning of the code stream? 

(3) Our example cited earlier seems to indicate the danger of a blocking due 
to inefficiency of the syntactical test: it is the constellation [A,, B,[= 
(0.011 ---1*,0.100---0* [| which looks dangerous for renormalization. Try 
to control the situation numerically. Will we need an algorithmic solution 
(exceptional case)? 


Remark In arithmetic coding, the sequence of source symbols s182--+ 8, will 
be encoded and decoded progressively; this allows us to implement an adaptive 
version of arithmetic coding which will learn progressively the actual proba- 
bility distribution p‘ (after the production of the first n source symbols). 
Let us make this more precise: at the beginning, there is no statistical 


information concerning the production of the N symbols ao, a1,...,a@n—1. We 
are obliged to put p = =(*> wD ae x) (uniform distribution). 


Now let p(™ = (ps, ae pom ,) be the actual distribution at the nth pulse. 
p‘” was computed at the last node nz of a control pulsation (ng < n) 

according to the histogram of sos1---s 
Let Sn41 =a; (O< 7 <N-1). 
Then we shall put: Ani. = An + (se 1 pi” )) (Bn — An), 


Bn4ai- An-+1 = pi m), (Bn 2S An). 


Nk 
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The decoder will recover 5,41, since it knows p(”) — the actual probability 
distribution after the production of the nth character — due to the histogram 
established with the information of anterior decoding. 


1.2 Universal Codes: The Example LZW 


The algorithms for data compaction which we shall treat now are “univer- 
sal” in the following sense: the idea of a memoryless source — which is a nice 
but very rare object — will be sacrificed. We shall joyfully accept source pro- 
ductions of a high statistical complexity; this means that explicit statistical 
evaluations of the source production will disappear. It is a new way of coding 
(by means of a dictionary) that creates an implicit histogram. 

The birth of the central algorithm Lempel-Ziv-Welch (LZW) dates to 
around 1977. There exist a lot of clever variants. We shall restrict ourselves 
to the presentation of a single version which is seductively clear and simple. 


1.2.1 LZW Coding 


Situation A source produces a stream of letters, taken from a finite alphabet. 


The encoder will establish a dictionary of strings of characters (of motives) 
which are characteristic for the source stream (and it will create this way 
implicit statistics of the source production). The compressed message is given 
by the stream of pointers (= the numbers attributed to the strings in the 
dictionary). Note that we meet here a method of coding where the code words 
have fixed length. 


The Encoding Algorithm 


The principal aim of the encoding algorithm is the writing of a dictionary: 


Strings of characters ~—> Numbers for these strings 


The production of the code stream (i.e. of the sequence of pointers) is 
(logically) secondary. 
The encoding algorithm works in three steps: 


(1) Read the next character x (arriving from the source). 

(2) Complete a current string s (which is waiting in a buffer for admission to 
the dictionary) by concatenation: s +> sa. 

(3) Write the current string into the dictionary as soon as it will be admissible 
(i.e. unknown to the dictionary) — otherwise go back to (1). Produce (i.e. 
send off) the number of s at the moment where you write sz into the 
dictionary; initialize s = x. 
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Remark What is the reason for this enigmatic delay at the output of the 
encoder? Let us insist: the encoder produces the code word of s at the moment 
when it writes sx into the dictionary. 

The reason is the following. The decoder will have to reconstruct the dic- 
tionary by means of the code stream which it receives (the dictionary of the 
encoder will not survive the end of the compaction). Without this delay be- 
tween writing and producing, the reconstruction of the dictionary by the de- 
coder would be impossible. 


Let us recapitulate the encoding algorithm in a more formalized version. 


STEP: read the next character x of the source stream. 


If no character x (end of message), 
then produce the code word (the pointer) of the current string s; 
end. 

If the string sx exists already in the table, 


then replace s by the new current string sz; 
repeat STEP. 

the string sx is not yet in the table, 

then write sz in the table, produce the code of s and put s = z; 
repeat STEP. 


IR 


Example Initialization of the dictionary: (1) 
(2) 
(3) 


a 
b 
% 


Encode baca caba baba caba. 


Read |Produce] Write Current string 

(1) a 

(2) b 

(3) c 
b b 
a (2)  |(4) ba a 
c (1) |(5) ac c 
a (3) | (6) ca a 
a (5) |(7) aca a 
b (1) |(8) ab b 
a ba 
b (4) |(9) bab b 
a ba 
b bab 
a (9) |(10) baba a 
; (7) |(11) acab a 
a ba 

(4) 
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Look, once more, how the source stream is transformed into the code 


stream: 
a c ac a ba bab aca ba 


(2) (1) (3) (5) () (4) (9) (7) (4) 


Exercises 


(1) The situation of the preceding example. 

Encode now baba caba baca caba baba caba. 
(2) A binary source. We initialize: (0) 0 

(1) 1 

Encode 010101010101010101010101. 

(3) (a) Give an example of an LZW encoding which produces a sequence of 
pointers of the type ...(#)(#)... (ie. with a pointer doubled). 
(b) Does there exist sequences of LZW code words of the type 
.. (H)(#)(#) ... (ie. with a pointer tripled)? 


1.2.2 The LZW Decoder 
A First Approach 


The principal goal of the decoder is the reconstruction of the dictionary of the 
encoder. It has to correctly interpret the stream of code words (pointers) that 
it receives. The down-to-earth decoding (the identification of the code words) 
is a part of this task. 

The current string s, candidate for admission to the dictionary, will still 
remain in the centre of the algorithm. Let us begin immediately with an 
example. 


Example Initialized dictionary: 


Decode (3)(1)(2)(5)(1)(4)(6)(6). 


Read|Produce}Write |Current string 

(1) a 

(2) b 

(3) ¢ 
(3) c c 
(1) a (4) ca a 
(2) b | (5) ab b 
(5) ab |(6) ba ab 
(1) a |(7) aba a 
(4) ca |(8) ac ca 
(6) ba | (9) cab ba 
(6) | ba |(10) bad ba 
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Attention We note that the two first columns are the result of a mechanical 
identification according to the input information of the decoder, whereas the 
two last columns come from a backward projection. What did the encoder do? 
Recall that the encoder can only write into the dictionary when appending a 
single character to the current string. Every word in the dictionary is preceded 
by the “dispersed pyramid” of all its prefixes. The steps of non-writing of the 
encoder disappear during the decoding. We write at every step. 
We observe: 


(1) At the beginning of every decoding step, the current string will be the prefix 
of the next writing into the dictionary. We append the first character of 
the decoded string. 

(2) At the end of every decoding step, the current string will be equal to the 
string that we just decoded (the column “produce” and the column “cur- 
rent string” of our decoding model are identical: at the moment when we 
identify a code word, the demasked string appears in the “journal” of the 
encoder — a consequence of the small delay for the output during the en- 
coding). 


The Exceptional Case 


Example The situation is as in the first example. Decode (2)(1)(4)(6). 


Read|Produce}Write |Current string 


(2) b 

(1) a (4) ba a 
(4) ba |(5) ab ba 
(6) baz |(6) bax bax 


Look a little bit closer at the last line of our decoding scheme. We have to 
identify a pointer which points at nothing in the dictionary, but which shall 
get its meaning precisely at this step. But recall our previous observations. We 
need to write a string of the form baz (current string plus a new character); 
but this must be, at the same time, the decoded string. On the other hand, 
the character « must be the first character of the decoded string. Thus, x = b 
(=the first character of the current string). 

All these considerations generalize: 

We have to identify a code word (a pointer) the source word of which is 
not yet in the dictionary. We proceed precisely as in the foregoing example. 

Let s be the current string at the end of the last step. Arrives the fatal 
pointer (4). If we take our decoding observations (1) and (2) as rules, then 
we are obliged to produce: sx write: (4) : sv put: s = sx, x is an (a priori) 
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unknown character, which will be identified according to the rule: c= the first 
character of the current string s. 

The question remains: what is the encoding constellation which provokes 
the exceptional case of the decoder? 


Exercise Show that the exceptional decoding case comes from the following 
encoding situation: the encoder sends off the last pointer available in the 
dictionary. 


The Decoding Algorithm 


Let us sum up, in pseudo-program version, the functioning of our decoder: 


STEP: read the next pointer (n) of the code stream. 
if no such (n), end. 
if the string of (7) is not in the dictionary, 
then produce sz, where z is the first character of the 
current string s, write sx into the dictionary, 
replace s by the new current string sz 
repeat STEP 
else the string u of (n) exists in the dictionary, 
then produce u, write sx into the dictionary, where s is the 
current string, and z is the first character of u, replace 
the current string by uw, 
repeat STEP. 


Exercises 
(1) Initialized dictionary: (1) a (2) 6 (8) c (4) d 
Decode (1)(3)(4)(3)(5)(7) (2) AD) (2) (4) (2) (6)) 7) (1) 
(2) Initialized dictionary: (1) a (2) 6 (8) c (4) d 
Decode (3)(1)(2)(6)(8)(4)(8)(7) 10) (5) (10) (7) (12) 11) (9) (1) (5) (14) (18) 
CN ee ee 


(3) A LZW decoder, initialized: (1) a (2) b...(26) z. 
At its 76th decoding step the decoder encounters an exceptional case. 
What is the number of this pointer? 
(4) Initialized dictionary: (0) 0 (1) 1 
(a) What is the bitstream which is encoded in (0)(1)(2)(3)(4) (5) (6)(7) (8) 
(9)(10)(11)? 
(b) Let us continue: we shall produce in this way all the integers, in natural 
order, from 0 to 65,535. 
Let us write the pointers in 16 bits. What will be the compressed bit-rate 


number of code bits 4 


P = ‘number of source bits’ 
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(5) The situation of exercise (4). The binary source producing a bitstream 
such that the stream of LZW-codes will be (0)(1)(2)... (65,534) (65,535). 
Model this source as a memoryless (!) source for the alphabet of the 256 
8-bit bytes 00000000, 00000001,..., 11111111. What will be (roughly) the 
source statistics? What will be its entropy? 

(6) Initialized dictionary: (1) a (2) 6 (8) c (4) d 
Decode (4)(1)(3)(6)(1)(5)(7)(7)(5)(5) (2) 1) (15) (14) (9) (19) (17) (1) (19 

(16) (19) (13) (22) (10) (20) (26) (25) (15) (33) (32) (35) (15) (24) (38) (2 
(7)(1). 


WN 
See Ne 
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Cryptography 


Cryptography is the theory (and the practice) of the computational aspects 
in the domain of data security: It deals with the design and validation of algo- 
rithmic methods which are meant to guarantee the integrity, the authenticity 
and the confidentiality in data transmission (and in data storage). 

The palette of cryptosystems is rich. On the other hand, the funda- 
mental algorithmic ideas in most of the currently canonized ciphers are 
almost exclusively the result of a lucky raid in elementary number the- 
ory. This has determined our selection. Thus, after having looked at Data 
Encryption Standard (DES — which is of a comforting mathematical simplic- 
ity), and having rather extensively treated the new data encryption stan- 
dard: the cipher AES-Rijndael, where a deeper mathematical understanding 
will be needed, we shall finally limit our attention to the investigation of a 
hard core of cryptosystems: The “arithmetical” systems, based principally 
on the idea of encryption by means of exponentiation in certain multiplica- 
tive groups. Beginning with a traditional and rigorously academic approach, 
we feel happy to present the (aging) system RSA, with which elementary 
arithmetic entered definitely and with unanimous acclaim into cryptogra- 
phy. Then, public key oblige, the chapter will hoist to a relatively broad 
exposition of digital signature algorithms; the algorithm DSA, the mathe- 
matical framework of which is relatively close to that of RSA, should prove 
exciting for all fans of technical details. We also supply the variant rDS'A 
and the algorithm ECDSA, which features elliptic curves — thus proving 
the practical interest of the purest mathematics. Finally, at the end of this 
ride across the “land of standards”, we will also have looked closely to the 
hash algorithm SHA-1, which is a sort of twisted parody of our old friend, 
the DES. 
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2.1 The Data Encryption Standard 


In 1973, the National Bureau of Standards of the United States invited tenders 
for a cryptosystem, abutting finally onto the DES — an amplified version of a 
former “family cipher” of IBM, called LUCIFER. After a long series of public 
discussions and controversies, DES was finally adopted as standard for data 
encryption in January 1977. Since, DES has been reevaluated almost every 
five years. The most recent version dates from 1994.1 


2.1.1 The DES Scheme 


The algorithm DES transforms plaintext blocks of 64 bits into ciphertext 
blocks of 64 bits. The cipher keys (attributed to the users) will consist of 56 
bits, in 64-bit presentation, with eight parity-check bits at the positions 8, 16, 
24, 32, 40, 48, 56, 64. The same algorithm is used as well for encryption as 
for decryption.” 


Global Structure of the Algorithm 


The algorithm works in three principal steps: 


1. The plaintext block T = tyt2--- tgq is first treated by a fixed initial permu- 
tation IP, which shuffles, in a highly regular manner, the binary positions 
of the eight 8-bit bytes. 

2. Then follow 16 iterations (16 rounds) of a transformation which depends, 
at each round, on another round key, extracted from the cipher key of the 
user by an auxiliary algorithm (the key schedule). 

3. The result of the 16 rounds has still to undergo the permutation [P7', 
the inverse of the initial permutation, in order to become the ciphertext 
block. 


' Omitting all preliminary definitions and generalities, we shall consider DES as an 
introductory example for cryptographic language. 
2 A strange statement — to be explained in a moment. 
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Scheme of the Algorithm DES 


plaintext 


IP 


Lo Ro 


— ky 


Li = Ro Ry = Lo © f(Ro, K1) 


<— Kj5 


Li5 = Ria Ris = Lia ® f( Ria, K15) 
J — Ki¢ 
Rig = Lis ® f (Ris, Ki6) Lig = Ris 
J 
IP7! 


L 
ciphertext 


The Functioning of the 16 Iterations 


Let JT; be the result of the ith iteration, 1<i< 16. 
T; = L( Ri = titg---tgots3 +++ tea 


DL, = Ri-1 
Then 
R, = y-1  f(Ri-1, Ki) 1<i<16 


where © means addition of vectors with 32 entries over the field F2 = {0,1} 
(in another language: we use 32 times the Boolean operation XOR). f is a 
“mixing function”, the first argument of which (the right half of the current 
state) has 32 bits, and the second argument of which (the current round key) 
has 48 bits. Its values will have 32 bits. 


Attention The result of the last iteration is RigL1¢, and 1P~' will operate 
on Riglie. 


Remark We immediately see that the DES scheme is “generically” invertible. 
More precisely: Every round is invertible — for any cipher key K and every 
possible choice of the “mixing” function f. You have only to write the round 
transformation scheme “upside down”: 


E1=R: Of, KK), Bi= hi, 1<i< 16. 
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This basic observation will give rise to an important construction in filter 
bank theory: The lifting structures. We shall treat this subject in the last 


chapter of our book. 


Exercise 


We consider a mini-DES of four rounds, which transforms 8-bit bytes 


%1X2°++Xg into 8-bit bytes y,y2---yg. We shall make use of keys Kr 


ujU2::- Ug of eight bits. 


L1LQU3UAL5 LEXLT LE 


L, = Ro R, = Lo © f(Ro, Ki) 


Lg= Ry Ry = Li © f(A, Ke) 


L3 = Re R3 = Ly ® f(Ro, K3) 


Ra = L3 © f(R3, Ka) L4= R3 


iPee 


Y1Y2Y3Y4Y5 Y6YTY8 


(a) [P= ( : : : s j : (assignment of positions) i.e. 


IP(x1%2%3U4U5U6U7Lg) = C5Lgl{L6l4U7 13X92. 


(b) The algorithm which computes the four round keys: K = ujug-:: 


will give rise to 


Ky = u7uiuzus, K3 = ujuaurua, 
Ko = uguguaug, Kya = ugusugus. 


(c) The mixing function: f(rirarsra, titetsts) = 21222324 , where 
24, =7T1 + te rhs + t4 mod 2, 
22 = 7Ta7 ty Tl 3 oF t3 mod 2: 
23 =T1 47 ty Ta. oe t4 mod 2; 


Z4 = TQ +1 ty 1 Fae t3 mod 2. 


— Encrypt « = 10011010 with A = 11001100. 


Us 
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The Symmetry of the Scheme DES 


For the decryption, we shall use the same algorithm as for the encryption, 
but with the order of the round keys inverted: Ky will parameterise the first 
iteration, K15 the second one, ..., Ky, the last one. 

Let us show that, indeed, Dk (Ex(M)) = M for every plaintext M (where 
Ex and Dx are the encryption and decryption transformations for the cipher 


key Kx). 

Observation: Dy-1 = R; @ f(Li, Ki) Ri-1 = DL; 1 < } < 16. 
We have 

C = Ex(M) =IP710 TA o TH o-..0 T™ o IP(M) 


M! = Dg(C) = IP 0 TK o pk O++0 pki o IP(C) 


We have to show: M’=M. 
First: 
M’' = bcm Sie ston oh eI PoiP or oi, oes) 
of P(M) = IP~10 TH! o TH? o+--0 TE (Rig, Lis) 
Then 
Th (Rie, Lis) = (Lie, Ris © f (Lie, Kis)) = (Ris, Lis), 
Ty (Ris, Lis) = (Lis, Ris © f(L1s, K1s)) = (Raa, Lia) 


T13?(Ro, Lz) = (Lo, Re ® f (Le, K2)) = (Ri, L1), 
Tis! (Ri, L1) = (Ri ® f(L1, Ki), Li) = (Lo, Ro) 
and we have finished... 


Remark The symmetry of the DES scheme, as we have shown it above, does 
neither depend on the size of the plaintext block, nor on the number of rounds, 
nor on the “mixing” function f, nor on the key schedule. We dispose of a broad 
range for variation. 


2.1.2 The Cipher DES in Detail 
1. The initial permutation IP: 
It is described by the following table: 


IP 
58 50 42 34 26 18 10 2 
60 52 44 36 28 20 12 4 
62 54 46 38 30 22 146 
64 56 48 40 32 24 168 
57 49 41 33 2517 9 1 
59 51 43 35 2719 113 
61 53 45 37 29 21135 
63 55 47 39 31 23 15 7 


You have to read line by line: IP(tite Sees testes) = tsst5o0 foe de tistr. 
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IP will shuffle, in a very regular manner, the bit-positions of the eight 
8-bit bytes which constitute the block T of 64 bits: every new byte contains 
precisely one single bit of every old byte. 

The inverse permutation [P7!: 


Tp 
40848 16 56 24 64 32 
39747 15 55 23 63 31 
38646 14 54 22 62 30 
37545 13 532161 29 
36444 12 52 2060 28 
35343 11 5119 59 27 
34242 10 5018 58 26 
33141 9 49175725 


2. The function f and the S-boxes: 


The first argument R of the function f is a 32-bit string, the second J has 
length 48, and the result f(R, J) will have 32 bits. The operations are orga- 
nized according to the following scheme: 


F(R, J) 


— At the beginning, the first argument of the function f is “stretched” into 
a string of 48 bits, according to an expansion table E. E(R) is made of all 
bits of R in a natural order, with 16 repetitions. 

— E(R) ® J is computed, and the result B is treated as a block of eight 
substrings made of six bits: B = B, B2B3B,B5 Beg B7Bs. 

— The following step makes use of eight S-bores 5S}, Sz, ...Sg. Every Sj; 
transforms its block 6; of six bits into a block C; of four bits, 1 <j <8. 
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— The string C = C,C2oC3C4C5CgC7Cy of length 32 is reorganized according 
to a fixed permutation P. The result P(C) will be f(R, J). 


The operation of expansion F is defined by the following table: 


E 


32 1 2 3 5 
45 6 7 9 
8 9 10111213 
12 13 14 15 16 17 
16 17 18 19 20 21 
20 21 22 23 24 25 
24 25 26 27 28 29 
28 29 30 31 32 1 


4 
8 


Read it line by line: = E(ryrar3+++ 132) = rgeT11%2 +++ 13211. 


Before explaining the functioning of the S-boxes, we shall present them all: 


Sy 
1441312 1511 8 3106125 907 
01574142 13110612119 538 
4 1148136 2 1115129 7 3 1050 
151282491 7 5113 1410 0 613 

So 
1518146113 4 9 7 2 13120 5 10 
3134 7152 8 1412011069115 
01471110 4131 58126 932 15 
1381013154 211671205149 

S3 
10 0 9 1463155 1138127114 2 8 
1370 934 6102 8 5 14121115 1 
1364 98153 0111 2125 1014 7 
11013069 8 7 41514 3 115 2 12 


S4 
7131430 6 9 10 1285 1112 4 15 
1381156150 3 47212 1 1014 9 
106 901211 7 181513145 2 8 4 
31506101 13 8 9451112 7 2 14 


Ss 
2124 1 710116 8 5 3 1513014 9 
14112124 7131 5 0151039 8 6 
4 2 1111013 7 8159125 630 14 
1138127 1142 136150 9 1045 8 
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411 2 141508 13 31297 5 
130117 491 1014 3 512 2 
1 4 1113123 7 1410156 8 0 
611138 1410 7 9 501514 


The functioning of the S-boxes: 

Consider a 6-bit string B; = b1b2b3b4b5bg. We shall compute a 4-bit string 
S;(B;) in the following way: 

The two bits b1bg are considered to be the binary notation of the index r 
of one of the four lines of S;, O<r<3. 

The four bits 62b3b4b; constitute the binary notation of the index s of one 
of the 16 columns of the table, O<s< 15. 

S';(B;) will be the 4-bit notation of the entry $;(r, s). 


Example $5(101010) = 1101. 


Remark The known criteria (made public in 1976) for the design of the 
S-boxes are the following: 


0. Every line of an S-box is a permutation of the integers 0,1, ..., 15. 

1. No S-box is an affine transformation. 

2. The modification of a single input bit of an S-box causes the modification 
of at least two output bits. 

3. For every S-box S and all B = b,bob3b4bsbg, S(B) and S(B & 001100) 
differ in at least two bits. 

4. For every S-box S$ and all B = _ 6,bb3b4bsbg we have S(B) #£ 
S(B @ 11a00) for a, G € {0,1}. 
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5. For every S-box, if an input bit is kept constant and if we are only inter- 
ested in a fixed output position, the number of quintuples which produce 
0 and the number of quintuples which produce 1 are near to the mean, 
i.e. they vary between 13 and 19 (note that whenever the fixed bit is b; 
or bg, there are precisely 16 values which produce 0 and 16 values which 
produce 1, according to the criterion 0). 


Finally, the permutation P (which seems to be meant as a remedy to the local 
treatment of the data): 


P 
16 7 20 21 
29 12 28 17 
1 15 23 26 
5 18 31 10 
2 8 2414 
3227 3 9 
19 13 30 6 
2211 4 25 


Read line by line: P(e, c9c3 +++ 32) = C16C7C20 +++ CaCa5- 


3. The key schedule: 


The cipher key K (of some user) is a 64-bit string, 56 positions of which 
constitute the real key, and eight positions support parity-check bits. The bits 
on the positions 8,16,...,64 are such that every 8-bit byte contains an odd 
number of ones. This way one can detect an error of a 1-bit fault per 8-bit 
byte. In the computation of the round keys, the parity-check bits are clearly 
neglected. 


1. Starting with the 64 bits of K, one skips first the parity-check bits, and 
permutes the key bits; the operation PC — 1 ( permuted choice) finally 
gives CoDo = PC — 1(K) where Co is the block of the 28 firsts bits of 
PC — 1(K) and Dp the block of the remaining 28 bits. 

2. Then, one computes 16 times 


Ci = LS; (Ci_-1), 
and K; = PC — 2(C;, D;). LS; is a cyclic left shift of one or of two positions, 
depending on the value of 2: it is of one position if 2 = 1, 2,9 or 16, otherwise 
it is of two positions. PC’ — 2 is another “permuted choice”, which has to skip 
eight positions in order to obtain finally 48-bit strings. 
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K 
PC—-1 
Co Do 
i 
LS, LS, 
J d 
C7} Dy, 
i J 
LS» LS», 
J i 
Cz Do 
1 a 
J J 
LSi¢ LS 46 
1 1 
Cie D6 


PC -2 ky 
POs Ks 
— |PC—2 — Ki¢ 


The two selective permutations PC — 1 and PC — 2, operating for the com- 
putation of the round keys, are: 


57 49 41 
1 58 50 
10 2 59 
1911 3 
63 55 47 
7 62 54 
14 6 61 
2113 5 


25 17 9 
34 26 18 
43 35 27 
52 44 36 
31 23 15 
38 30 22 
45 37 29 
20 12 4 


14 17 
3 28 
23 19 
16 7 
41 52 
30 40 
44 49 
46 42 


24 1 5 
6 21 10 
4 26 8 
20 13 2 
37 47 55 
45 33 48 
56 34 53 
36 29 32 
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Exercises 


(1) A mini-DES with four iterations, transforming 8-bit bytes into 8-bit bytes. 
The keys will have also eight bits. 


(a) The initial permutation: [P(x1%203%4%5%6U70g) = Cgl5LQL7L4U UL6L3. 
(b) The key schedule: Let K = uj ugu3zuqusugu7ug, then Ky = uguyuzugusuz, 
Ko = ugugu2u7zugug, K3 = ugugu,ugu7us, Ky = usuguguyugua. 
(c) The function f(Rj-1, .K;): 
Compute first the binary word 6,b2b3b4b5be from Rj; = rirar3r4 and 
K; = ty tatstatste in the following way: 
6, b2b3b4b5 bg = P11Tar3grgri1r3 @ tylotgtatste 
(®: Boolean XOR, componentwise). Then, we shall use the following S- 


nN 


box: 

0123 456 7 8 9 101112131415 
04112 141508133129 751061 
1130117 49110143 5122158 6 
21 41113123 71410156 8 059 2 
361113 8 14107 9 5 01514 2 3 12 


61 b2b3b4bs5bg points at a line b,bg and at a column b2b3b4bs of the S-box. 
f(Ri_-1, Kj) will be the 4-bit notation of the integer which occupies the 
entry. 

(d) Encrypt M = 00100010 with AK = 01110111. 


(2) Now we deal with usual DES. 
The plaintext: M = 8aflee80a6b7c385. Write down the fifth 8-bit byte of 
IP(M). 

(3) Let B = Ofe3le4fb0f0 (48 bits; you have to think in 8 times six bits). 
Find $(B) (32 bits — in hexadecimal notation). 

(4) The plaintext: 00 00 00 00 00 00 00 00 (64 zeros). 
The cipher key: 01 01 01 01 O1 O01 01 01 (56 zeros — the ones are the 
parity-check bits). 
(1, Ri) = ? 

(5) Let c(.) denote the operation of complementation for binary words (p.ex.: 
c(0110) = 1001). 
Let y = DES(a, K) be the ciphertext obtained from the plaintext block 
x and the cipher key K. Show that c(y) = DES(c(x), c(K)) (complemen- 
tation of the plaintext and of the key will yield the complement of the 
ciphertext). 

(6) The round keys may be obtained from the cipher key of the user by means 
of “selection tables”. Find the selection table for Kg. 


60 2 Cryptography 


Solution 


K6 
3 44 2717 42 10 265060 2 41 35 
25571918 1 51 52 59 58 49 11 34 
13 23 30 45 63 62 38 21 31 12 1455 
20 47 2954 6 15 4 5 39 53 46 22 


(7) Let K = abababababababab (64 bits) be your cipher key. Which is your 
sixth round key Kg (48 bits, in hexadecimal notation)? 
(8) Differential cryptanalysis — level 0. 


S| 0 12345 6 7 8 9 101112131415 
0 |14 4 1312 15118 3106125 9 0 7 
1/015 74142131106 12119 5 3 8 
2/4 114813 6 21115129 7 3 105 0 
3 |1512824 9 1 7 5 11 3 14100 6 13 


Notation The inputs: B, B*, etc. (six bits) 
The outputs: C, C*, etc. (four bits). 


(a) For an input modification B’ and an output variation C’ of (51) 
let support(B’ — C’) = {B : S\(B @ B’) @ S\(B) = C’}. Find 
support(110100 —- 0100) — a set of two elements. 

(b) Observation: The first six bits of the round key Ky are the bits kyo, 
ks1, kza, keo, kao, kyi7 of the cipher key. Let. Ro = 00001---0 and 
Rj = 10101---1 be two chosen “plaintexts” . 

We observe: The two outputs of the S$ -box C' and C* ( coming from Ro 
and from R) differ only at the second position (C6 C* = 0100). 
Find ky7, k34 and k4g (we shall make use of the result of (a)). 


2.2 The Advanced Encryption Standard: The Cipher 
Rijndael 


It was the cipher Rindael, designed by Vincent Rijmen and Joan Daemen, 
which was the surprise winner of the contest for the new Advanced Encryption 
Standard of the United States. This contest was organized and run by the 
National Institute for Standards and Technology (NIST), at the end of the 
1990s. In December 2001, the cipher Rijndael became the successor of DES, i.e. 
the AES = Advanced Encryption Standard. Much more soaked in Mathematics 
than its predecessor, the system AES remains altogether in the tradition: you 
will find S-boxes, rounds and round transforms, and round keys. 


2.2.1 Some Elementary Arithmetic 


The letters of the system AES will be 8-bit bytes, with an intrinsic arithmetic 
that stems from their identification with the 256 elements of a field of residues 
(modulo a certain irreducible binary polynomial of degree 8). 
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In order to be perfectly at ease with this kind of arithmetic, we shall recall 
the essential facts about the most frequent specimen of quotient rings, first 
the integer case (i.e. we shall treat the quotient rings of Z), where everything 
is rather natural (and familiar ...), then the case of binary polynomial arith- 
metic, where the algebraic skeleton is simply obtained by formal analogy with 
the integer case. 

Note that the quotient rings of these two Fuclidian rings — the ring of inte- 
gers and the ring of binary polynomials — are omnipresent as algorithmic tools 
for information security (in cryptography as well as in theory and practice of 
error control codes). 


Recall: The Arithmetic of the Rings of Residues Z/nZ 


Let us keep n > 1 fixed. Definition of an equivalence relation on Z: 
x =ymodn <=> « and y leave the same remainder when divided by 
n <> «-y is divisible by n. 


Notation [z]=2modn for the equivalence class of x. x mod n will also 
denote the canonical representative of this class: the (non-negative) remainder 
when dividing x by n. 


Fundamental Observation 


The so-defined equivalence relation is compatible with addition and with 
multiplication on Z: 


x=x'modn crty=2'+y' modn 
y=y' modn xv-y=u'-y' modn 
Consequence: 
The set Z/nZ of the n equivalence classes {[0], [1],...[n —1]} bears a natural 


ring structure (of a finite commutative ring): 


[z]+[yJ=[e+y]  [e]-l=le-gl, 
where [0] is the zero element for addition, [1] is the unit for multiplication. 
We have, of course: —[%] = [—2]: the opposite class is the class of the opposite 
representative. 


Example Z/12Z = {[0],[1],...[11]} 


Let us look closer at the following products: 


[1]- [1] = [1], [2] - [6] = [0], [3] - [4] = (0), 
[5]-(5]= (1), [7-7 =(4, [8] - (3) = (0), 
[9] - [4] = [0], [10] - [6] = [0], [11] - (14) = [1] 
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We note: — [a] is invertible for the multiplication in Z/12Z <=> ais 
prime to 12 (i.e. @=1,5,7,11 mod 12). 


Let us turn back to the general case: 


Wedenoteby (Z/nZ)* the group of multiplicatively invertible elements 
in Z/nZ. 


Proposition 
[a] € (Z/nZ)* <= > — gcd(a,n) = 1. 


Proof [a] € (Z/nZ)* <= there exists x € Z: |[a]-[z] = [1] © there 
exist z,yE€Z: axr-l=ny Ss theequation aX+nY=1 = admitsa 
solution (x,y) € Z x Z. 

We still have to show: 

The equation aX+nY=1 admits a solution (1,y)€ZxZ <— 
gcd(a,n) = 1. The reasoning goes as follows: 

On the one hand, a common factor p of a and n must divide azx+ny, 
which we suppose to be equal to 1; on the other hand, if gcd(a,n)=1, then 
we compute a solution (2,y) € Z~x Z for the equation aX +nY =1 by 
means of the Euclidean algorithm which determines the gcd(a,n) (=1). 


Example Compute an integer solution of the equation 37X + 100Y = 1. 
Let us recover gcd(37, 100) = 1 by iterated Euclidian divisions: 


100 = 2-374 26, ie. 100=2-37+17), 
37 = 1-264 11, 387 =" +712, 

26 =2-11+4, Ty =2-ro+ 73, 
11=2-4+3, rg =2-7r3 +14, 
4=1-3+1, rg =ra+1. 


Now, we have to eliminate the remainders: 
Ll=r3—1r4=13 — (re — 213) = rg — rq = 3(r1 — 2re) — re = 38r1 — Tro —_ 
3r, —7(37 —1r1) = 10r, — 7-37 = 10(100 — 2-37) —7-37 = 37-(—27) + 100-10. 
Our solution: (x, y) = (—27, 10). 
Once more: [37]~! = [-27] = [73] in (Z/100Z)*. 


Corollary Z/nZ is a field (every non-zero residue is invertible for the 
multiplication in Z/nZ) <=> nis a prime number. 


In this case en=pisaprimenumbere (Z/pZ)* is a cyclic group of 
order p — 1: 

There exists an element w € Z/pZ_ such that the list of the p—1 powers 
w, w*,...w?-! = 1 contains precisely all the elements of (Z/pZ)*. 


An element w with this property is called primitive modulo p. 
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Example (Z/17Z)*: 


w=3, we=9, we =10, wt=13, 
w? =5, oa 1h, S411, we H16; 
eat oi aR pte lt A), 
west CoS. rag. gis. 


Let us insist: If p is a prime number, then the multiplicative group 
(Z/pZ)* is acyclic group of (p—1)th roots of unity, i.e. it can be written as 
(Z/pZ)* = {w,w,...,wP? = 1}. 


In particular, for every divisor n of p — 1,the group (Z/pZ)* contains a sub- 
group I), of nth roots of unity: 


p-1 p-1l 


p-1 
Tn = {w Do pice ad oe oie 


Remark Starting with a fixed prime number p, the smallest (positive) integer 
which is a primitive root modulo p is not obtained in a natural way: Take, 
for example, the following five consecutive prime numbers, and the associated 
(minimal) primitive roots: 


p = 760301 : Ww = 2, 
p = 760321 : w = 73, 
p = 760343 : w=5, 
p = 760367 : w=5d, 
p = 760378 : w= 2, 


By the way, already the candidate 2 gives rise to a remarkable problem: One 
does not know, whether 2 is a primitive root modulo p for an infinity of prime 
numbers p. 


Exercises 


(1) Find the smallest positive integer w which is a primitive root modulo 
p = 3,5,7,11, 13,17. 

Decide, in all cases, if w remains primitive modulo p? (w will then remain 

primitive modulo p* for every k > 2). 


The following exercises deal with the important Chinese Remainder 
Theorem. 


(2) Show the Chinese Remainder Theorem for the following particular case 
(which is perfectly generic. . .): 


Let N =n n2n3 be a product of three (mutually) coprime integers. Then the 
natural mapping 
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aw: Z/NZ > Z/nZx Z/noZ x Z/n3Z 


xmod N + > (a mod 1,2 mod no, x mod n3) 
is an isomorphism of rings. 


Solution Since the source and the target set have the same cardinality N = 
nyngn3, we have only to show the surjectivity of 7: 

For given 21,%2,%3 € Z, there exists x € Z with x = x, modn, « = 
Yq mod nog, x = 73 mod nz. 


We search x as a linear combination x = a,2%1 + agu%q + a3%3 (where ay, 
az, a3 will not depend of 21, x2, x3, i.e. they will define the inverse mapping 


iad 
We need: 
a, =1modny, a, =0 mod no, a, =0 mod nz, 
az =0 mod ny, a2 =1 mod no, a2 =0 mod nz, 
a3 =0 mod ny, a3 =0 mod no, a3 = 1modnz, 
a, = ayn2n3 a1 = (n2n3)~! mod ny, 
We find: a2 = N1A2QN3 with a2 = (nin3)~! mod no, 
a3 = N1N2A3 a3 = (nyn2)~' mod nz. 


(the multiplicative inverses exist precisely because of our coprimality hypoth- 
esis ON 11, 72,73). It is clear, that everything generalizes to coprime decom- 
positions of arbitrary length. 


(3) In the situation of exercise (2), let N = 1001, with nj = 7, ng = 11, 
ng = 13. 


Make explicit m1 : Z/7Z x Z/11Z x Z/13Z. =—> Z/1001Z 
(i.e. find a1, a2, a3). 


(4) Resolve the following congruences (separately): 


3x = 4 mod 12 27x = 25 mod 256 
9x = 12 mod 21 103x = 612 mod 676 


(5) Show that n° — n is always divisible by 30. 


(6) Find the smallest non-negative solution for each of the following three 
systems of congruences: 


(a) «®=2mod3 
(b) «=12mod3l1 
x =3mod 5 (c) 19x = 103 mod 900 
x = 87 mod 127 
x =4mod 11 10z = 511 mod 841 
x = 91 mod 255 
x =5 mod 16 


2.2 The Advanced Encryption Standard: The Cipher Rijndael 65 


The Arithmetic of the Field of Residues F2[]/(p(x)) 


Let F2[z] be the ring of polynomials with coefficients in the field Fz = Z/2Z. 
This is an Euclidian ring (we dispose of the Euclidian algorithm), exactly like 
Z,the ring of integers. 

As a first step in analogies, we shall be interested in “prime numbers” for 
F.[a]. 


Definition p(x) is an irreducible polynomial <> p(x) has no non-trivial 
divisor (i.e. 1 and p(x) are the only divisors of p(x)). 


Example The irreducible polynomials of degree < 4: 


(1) degree l: a2, a+1 

(2) degree 2: 2?+a2+1 (note that 2?+1=(xr+1)?, since 1+1=0) 

(3) degree 3: a +a4+1, «3 +2?+1 (a cubic polynomial is irreducible 
<=> it has no linear factor <=> it admits neither 0 nor 1 as a root) 

(4) degree 4: wt+at+i1, at+a°941, ztt+a%t+e?+e41 


(the absence of a linear factor forces the polynomial to have a con- 
stant term = 1 and an odd number of additive terms; on the other hand: 
(2?7t+a+1)% =24+27+1). 


Exercise 


Show that the binary polynomial p(x) = 2° + «4 +23 + 2? + 1 is irreducible 


(this is an exercise in Euclidian division: you have to test whether p(x) is 
divisible by any of the irreducible polynomials of degree < 4). 


Our next step: the rings of residues F2{2]/(p(z)). 
Let p(x) € Fela] be a fixed polynomial . 


The ring of residues (for the division by p(#)) Fa[x]/(p(a)) is defined in 
precisely the same manner as in the case of the integers. 

We note: If the degree of p(x) is equal ton, then Fa2[x]/(p(x)) admits 
2” elements: there are 2” remainders for the division by p(x) (i.e. there are 
2” binary polynomials of degree < n). 

Let us look at a simple but instructive 


Example p(x) =a*+a2+1. 
Frl\/(et+e+1)={ [0], (el, +, b,. 2+], +a, 


[ete T), (ae?) [a1], [ePal, [wt toetd), [e+e], [et+eb dl], 
[ee+a?+a], [x8 +a?+24+1] }. 


The 16 binary words of length 4 will be coded in remainders of division 
(by p(x) = x+ + 2 +1) in the following way: 


0000 << [0], 
0001 — [1], 


0010 ~~ [a] 


111 6 [ak+a2?+4+a4+1]. 
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The addition of residues is simple: 


[° + 1] + [a? + 2° + 1] = [a], 
1001 @ 1101= 0100 
(addition will always come from the Boolean XOR, position by position). 


As to multiplication, one should — in theory — proceed this way: 


(2? +a 1) + (2? +27] = [2 + 2° +074 27). 


Then, one searches the remainder of division (by «4 + x + 1): 


cota? +24 +a? : (24 +a4+1)=2?4+e41 


x ve +a? 
oe +at4 23 
5 2 
z i lng ia 3 
e+e +a° +e 
xt +a2+1 


This yields: [3 +a2+1]- [22+ 27] = [a3 +274] 
1011 -1100 = = 1101. 


How to avoid these (boring) Euclidian divisions when computing products? 

We observe: [x] = [x + 1] (since [z+ + x +1] = [0]). Let us give up the 
bracket notation (i.e. we replace equivalence by equality): 

rt=xt+l 

We can continue: 

P=2r*+ea, 

xe = 2? + 2?, 

Finally: 

ro +a°4+a4+4+ a7 = (23 +27) + (a2 +2)4+ (2+ 1) +2? = 23 +2741, as 
expected. 


Remember, for the general case: 

Let p(x) =2"4+ Bp-1a""- 1+... 814+ Bo € Fo[z]. Then the arithmetic of 
residues modulo p(x) (the arithmetic in F2[z]/(p(x))) will obey the following 
rules: 


(1) The addition of remainders is done “coefficient by coefficient”, respecting 
1+1=0. 
(2) The multiplication of remainders is done in two steps: 


(i) Multiplication of the remainders as simple binary polynomials. 
(ii) Reduction of the result to a remainder (i.e. a polynomial of degree < n), 
according to the reduction rule 2” = 6,-,a"~!+-+-+ Ba 4+ Bo. 
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But back to F9[2]/(2+ + 2 +1): 
Let us consider the successive powers of w=: 


w= 2, w® = 23 4 2, wit = 4 e%+2, 
w? = 27, w =e +e4+1, wl =o? +2? +241, 
is? =a, ws = 27 +1, wis = 34 a7+1, 
wo, wi = gi +z, wi4 = 3 +1, 

w =g+2, wl = 7+ e+1, wih =] 


We remark: The powers of w=2 _ yield a complete list of all 15 non-zero 
remainders in Fo[z]/(z* + 2 +1). 


F2[a]/(2* + a +1) is a field (of 16 elements) 


Every non-zero residue admits clearly a multiplicative inverse: 
(23 + a?)-! = (W8)-1 sw Hu Ha 42. 
For the incredulous reader: 
(2? +27)\(22 +2) = 2° +2°+a44+23 = (a? +07) 4 (2? +2)4+ (241) 4+2°=1 
But back to the general situation: 


Proposition The ring of residues Fa[a]/(p(x)) is a field <= > p(x) is an 
irreducible polynomial. 


The argument is the same as in the case of the integers: You have to 
use the equivalence: [a(x)] is invertible in the ring Fo[z]/(p(z)) <> 
ged(a(x), p(x) = 1. 


Complement Let p(x) =e? +---+1 — be an irreducible polynomial of 
degree n. 

Then the multiplicative group G = (F2[a]/(p(a)))* ts a cyclic group of 
order 2"—1 (this is a non-trivial fact ...). If w = x is a generator of 
this group (i.e. if the powers x*, 1 < k < 2” —1, produce all the non-zero 
residues modulo p(x)), then we say that the (irreducible) polynomial p() is 


primitive. 


Exercise 


Show that the polynomial p(x) = 24 +a°+2?+2+1. is irreducible, but 
that p(x) is not primitive. 
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Some irreducible and primitive (binary) polynomials: 


x+a+1, at + Ty a 42% 4a44 e041, a 4e° 41, 
e+a+l, et+at+a%+a741 gi ct+a%°4+ae4+1, ot 4e741, 

at +241, oe tatt Ts avl4 gO } a x+1, et 4 oP 4 a+at+l, 
a> +27 +1, v9 4 23 +1, gid +2a2+1, 2 +4 a3 +1, 

a +e4+1, vtt 4 a7 +1, oo 4 ola ee tet. 


Remark Let p(#) = «"+---+1 bea primitive (irreducible) polynomial. Then 
w = is a primitive (2” — 1)th root of unity in F2[z]/(p(2)) 


w-1=1 and w* 41 for 1<k<2"-1. 
For every divisor N of 2”—1, put N’ = eet. Then wy =w 
Nth root of unity in Fo[2]/(p(2)). 


N" is a (primitive) 


Notation Fan = Fo[2]/(p(x)) (p(%) is a distinguished irreducible binary 
polynomial of degree n — the notational laxity is partly justified by the fact 
that all finite fields of the same cardinality are isomorphic.*) 

Let us fix the notation (and the corresponding arithmetic) for the sequel: 


Fy = Z/20 = {0,1}, 
‘4 = Fa[a]/(2? +2 +1), 
Fg = F2[a]/(22 +2+1), 


gtt+a+l 


( ) 
(x +a” +1) 


? 


% 


Exercises 


We shall begin with the field Fo, = Fo[z]/(a® + x +1), then turn to the 
field Fo5g = Fo[x]/(a® + x* + 23 + x? +1). We shall always identify binary 
words (of length 6 or 8) with remainders of division, according to the scheme 
100101 = x + x +1 in Fea, 00100111 = x + x? +a+1in Fo56. 


(1) Write down the table of the 63 non-zero residues in F¢4, in function of the 
successive powers of w = we3 = &. 


3 There are 30 irreducible binary polynomials of degree 8, hence 30 realizations 
for “the” Galois field of 256 elements in terms of residue-arithmetic. In the fol- 
lowing exercises, we shall compare two of them. The reader will rapidly note the 
important differences in down-to-earth computations. 
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Solution The cyclic group of 63th roots of unity (F¢4)*: 


ce = e+ a> =2°>4+2, eta tei te? td 
e=a? +2, 2% =e? +a41, a =e744+ e541, 
egaeiaz a =e +272 +2, ec = 27> 4+a4+42, 

2 =at 423, 228 = 24423 4 2? et =e te tad, 
xl = 2 4 24, 22 =e +244 23, 28 = 23 +22 41, 

cian +r4+1, eae +et+e41, ec =et+e% +2, 

x? = 2241, et =p 42 4 iv 750 = 5 4 ot 4 9? 

a8 =e73 +2, a? = 273 +1, etoeg +a34+ae41, 

git Spt a: x, a x, eo? = xt +o? 4 1, 

xl = 2 4 23, x4 = 2 4 2, 23 — 2 42342, 

ot =e7t+e41, o> =e +241, et aatter+eH41, 

a => 4+a2 +2, po = 2442242, 2 — 75 49342242, 

et =e +2? 4+ae41, P= 7> 4+a54 27, oS =7tta 2 t+ae2?4+a4+1, 
i? at 4+ 23 tar tea 8 at + e+e+l, 2? = 4 ot + 2 + a? +a, 
2 =o 404423 +22, ee =e 4e442? +2, 2 = 4et4+e3 4024041, 
eiag4et4t a3 txe+1, 79 = ® 2 | a? x+1, 79 = o> } at 2 + x? +1, 
ee =e 404 4 oe? 4 1, wt! = ot 473 4 x? 1, 2 =e 4+ e44e3 41 
c= 27> +25 1; a= o> +a44 23 Ly St oe +at + 1, 

e4 = gt + 1, a3 = g® at | a? x+1, 78? = 7? + 1 


(2) (The discrete logarithm problem in F¢q): 


Let a = 11100, 
@ = 10101. 
Find the unique exponent e , 0 < e < 62 , such that 
a® = 6 mod (2° + 2+ 1) 
(Notation: e = log, 3) 


(3) Write down the table for the 255 non-zero residues in F256, in function of 
the successive powers of w = wo55 = 2. 


Solution The cyclic group of 255th roots of unity (F256)*: 


a 
© 


8 4 3 2 1 


ea=aet+art+e 3 gl? = go gt to? +2, 
eP=24+e4+e3 +2, £9 =e 4g +44 + 2, 

710 = 76 4 5 4 ot 4 9? at = 96 4 o5 4 a4 4 9? +1, 
a =a? 4 96 495 4 93, raat deft etarta, 
aaa? too 403 40? +1, 3 =a? 4 e% 49341, 

ga e=e2"4+e7+4+24+1, 4 =e +e3 4074241, 
a4=et+e4+1, x =—a¢+1, 

a5 =e + e242, asl = a7 4 26, 

16 = 96 4 93 4 o?, eae tet tad 4a? +1, 
at mal taett 23, 28 =o +e? 4241, 

ple gh et ce gt a4 = 96 4 e3 49242, 
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ge — ag? + go teh 4 ott eg +2 +1, 
27 gl tg +e +o tat, 
e236 — og? 47S 493 4941, 

ret gta tet, 

gr 8 =o + e+, 

att? eT 4a? ot 

8 — 7 t+e5 +a44 2342741, 
45 = oT 4 9 4 8 4 8 41, 

246 — og? 4 go 43 4g? + 41, 
ett al tat 1, 

8 —gtt oe t+etl, 

20d we gl ga gh ad 


ey) ee ere +a? 41, 
e723 = 7S + x “2+, 

o74 — 7? 4+ 93 407% 4+2, 

255 =] 


Solve the following two linear systems (over Fo5¢ = Fa[x]/(2° + a4 +23 + 


x? +1)): 
eto. (e+ 1)T, + 13 = 1, 
To xT; (a 1)T> T3 _ 0, 
To T; t rT (a t 1)T3 a 0, 
(a 1)To t T, t T» &T3 = 0. 
x®°Ty +4 e287, + 8? T, ~ 67, = x? , 
x°?To oT, e287, eae x'8?Ts = x, 
LTp xv?T x°°To x87, = l6 
x*6T) aT, xu”? T» 2°°Ts = 7158 
Show that (2 +1)5! =1 mod (2° + 24+2° + 2? 4 


Consider the involutive automorphism @: F9{z] 


Put p(x) =a +a4+4+ a2 +2 
mx)=28+a4+a23+2+1 


Show that D(p(x)) = m(z). 


Deduce a natural isomorphism ¢ : Fos5g¢ = Fa[a]/(p(2)) 


e+. 


i 


Fa[2]/(m(x)), # 


1) 
— 
ed 


x 


— Rose = 


Show that the polynomial m(x) = «8 + 24 +2? + a+ 1 is irreducible, but 


that m(a) is not primitive. Find the order of x in R356 = F2[z]/(m(z))*, 


* 


i.e. find the smallest positive exponent e such that 7° = 1 mod m(z). 
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7) Compute the following multiplicative inverses in Ras5g = Fo[x]/(m(a)): 
g 


(a) 11101010-! =? 
(b) 01010101-! =? 
(c) 11111111-! =? 


(8) The reference code in C of Rijndael contains two tables to support multi- 
plication *) in Rase = Fo[a]/(a8 + a+ + 22 +241): 


word 8 Logtable [256] = { 

0, 0, 25, 1, 50, 2, 26, 198, 75, 199, 27, 104, 51, 238, 223, 3, ---}: 
word 8 Alogtable [256] = { 

1, 3, 5, 15, 17, 51, 85, 255, 26, 46, 114, 150, 161, 248, 19, 53, ---} ; 


(a) Find the last four entries of every table. Justify. 


(b) Replace now Ras6 by Fosg = Fa[z]/(28 + at + 23 + x? + 1). 
Write the first 16 and the last 16 entries of each of the two tables, now for 
the arithmetic of Fo5¢. 


*) word8 mul(word8 a, word8 b) { 
/* multiply two elements of Ros¢ 
*/ 
if (a && b) return Alogtable[(Logtable[a]+Logtable[b])%255); 
else return 0; 


(9) Write down the table of the 255 non-zero residues in Ros¢, in function 
of the successive powers of €=a2+1. 


Solution the cyclic group of 255th roots of unity (Ras6)* 
2 = 9? 41, 10 — gS 4 Pa gta a, 
Bae +e2+e41, fl ae? tatters, 
*=a*+1, éPa ei te? 4-1, 
a ee ey se ray wee er er 
a re ek a cee e 
(eee ean re ore, per er ee eee, Gb SP At he, 
és Pa at pa L, été = 7® at pia a? xe As 
a ee es EVE gh Oil oe ere 


a ok Ro 
ge WN 


RR KR OR 
oO oN DD 


RN SSI LE ART EIN IE ET SEP 
oO 
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etatt x, 
eo + x, 
eot+a®t+atte?, 


ce +a°t+aett+aet1, 


ve +at*t+a? +1, 
5 

ve +a° +27, 

Tj. Ob ie lB: pp one 


=a 4+a°+e°+att+ar? tea 
=, 

=e +42, 

=e +42, 

=ae'+a%4+e7 42, 
=x2°+%, 

=7+a°4+27% +2, 

=e’ +a°4+a°4+2, 

=e 4+e% +e? 42741, 
=r4+at4 27, 

=e +a44+2°4 27, 

=e 4+a%4+2°4 27, 
=r4att+e?4+e41, 
=r +at4+ 2°41, 

=a +e%+e°+e2 +241, 
=e4a74+2, 
=r+a°4+2° +2, 

=a’ +artattarte? +a, 
=e 4+e%+at+4e7°4+1, 
=7°+2°4 24, 

=r 42%, 

=e 4+e° +22 4e41, 
=e +22 42° +07? +2, 
=e74+a*+1, 
=r°+at+e4+1, 

=a’ +a tar? tatte? +1, 
= 2’, 

=2° 427, 

=a*427, 

=e4atta% 427, 

= 7° +27, 

=e 4+a%4+ 22427, 

=r +a2 +2727 4241, 


=a +2°4+a74+23, 

=a’ +e%+at4+e4+1, 
=e +e°+a%4+a7% +a, 
=a +a°4+a*4+2, 

=a +ae%+a3 4+ a7 41, 
=a +a°427, 

=a +2°+a44+2?, 

=r +e°4+a7%+4+e41, 
=e 4+e°4+e2° +1, 

=a 4+a°4+2°, 

=e tatt+e2t+e+l, 
=r +a%4+27 +1, 

=a +a%+atte?+e41, 
=a +2 +2, 

=a +a°4+27%4+2, 

=x +a%+e? +0741, 
=a2'4+2°, 

=a4+2°, 
=a7°+a°4+a74+2°, 

=a 42°, 

=x’ +a+4+1, 

=a +at+e% +a? +2, 
=a +a°+at4+e°4+1, 
=a 4+a°4+24, 
=r+e°4+e2?4+e41, 
=a +a? tattae +a? +1, 
=a +a°+at4+a° 42? 
=a +a tatt+a% +a? +a 
=e’ +1, 

=a 4+a*4+2°, 

=a’ +a°+at*+aet+1, 
=a +e°+e%+a7 +2, 
=r +a° +1, 

=a +a%+att+e°+e41, 
=e +e°4+at*4+a7% +a, 
=a +a*4+a°4+2, 

=a +a°+at4+e7? +1, 
=a +a°427, 

=r +att+e?4+e41, 
=e 4+e°+e? 4+e44+ 22 41, 
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2.2.2 Specification of Rijndael 
Generalities 


AES encrypts a plaintext block of 128 bits (of 16 bytes) into a ciphertext 
block of 128 bits (of 16 bytes). 

AES exists in three versions: 

AES-128: the cipher keys have 128 bits, 

AES-192: the cipher keys have 192 bits, 

AES-256: the cipher keys have 256 bits. 

10 (AES — 128) 
The plaintext block undergoes 12 (AES — 192) iterations of a 
14 (AES — 256) 
transformation which depends each round on another round key extracted 
from the user’s cipher key. 

Note that the round key K;, 0 <i< Nr(= the number of rounds) will 
have 128 bits, i.e. it will have the same size as the message block during the 
encryption process. 

The round keys are produced by an auxiliary algorithm - the key 
schedule — the input of which is the cipher key of the user, exactly as in 
the DES. 


Remarks on Formal Questions 


The letters of the cipher AES—Rijndael are the (8-bit) bytes 


e Syntactical notation: 10100111 
e Hexadecimal notation: a7 (1010=a 0111 =7) 
e Polynomial notation: a +2°+a?+a+41 (this is an element of the 


field Ras6, ie. a remainder of division by m(x) = a8 + a4+a°+a+1). 


Summing up: 37(¢6--- 3180 = {(G7--- Ba)16(G3 --- Bo)16} = Bra? + Gor®+- +++ 
Bix + Bo 


External Presentation and Internal Presentation of a Block of 128 Bits 


The external presentation of the 128-bit plaintext block (and of the ciphertext 
block) is sequential: 

ao = BoP +++ Br 

a1 = Bg Bq --- Bis 


BoP182-+: Gie7 = a0a1a2°-- a15 with 


a15 = Pr20P121 eee Bi27 


Caution In individual treatment, every 8-bit byte will have a decreasing 
indexing of the positions: 7.6.5.4.3.2.1.0. 
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The internal presentation of the 128-bit message block during the encryption 
(or of the ciphertext block during the decryption) will be in matrix form (and 
will be called a state or state array). 


ag a4 ag 12 $0,0 50,1 $0,2 50,3 
a1 45 ag 413 = $1,0 $1,1 $1,2 $1,3 
a2 46 G10 414 a $2.0 $2,1 $2,2 $2,3 
a3 G7 A11 415 $3,0 53,1 $3,2 53,3 


Every state is a quadruple of (vertical) words, each consisting of 4 letters (4 
bytes): 


ao 50,0 a4 50,1 
a1 $1,0 a5 $1,1 
Wo — — ; Wy = —! 
a2 $2.0 a6 $2.1 
a3 $3.0 a7 $3,1 
S 
ag $0,2 a12 ae 
1,3 
— ag _ $1,2 — 413 _ 
W2 = = W3 = = $2.3 
a10 $2,2 a4 5 
3,3 
Q11 53,2 15 


In the sequel, we only consider the version AES-128. 


The Round Transformation 


The round transformation is composed of four constituent transformations, 
called the steps: 


— The S-box: It consists of 16 identical copies of an S-box acting on letters; 
the 16 bytes of the state are treated independently. 

— The ShiftRows step: This is a byte transposition that cyclically shifts the 
rows of the state over different offsets. 

— The MixColumns step: The four columns of the state undergo simultane- 
ously a circular convolution (as elements of R4;,) with a simple impulse 
response. 

— Addition of the round key: The round key adds to the state (addition of 
two 4 x 4 matrixes over R256). 


The S-box 


The S-box is the non-linear component of the round transformation (exactly 
like the Sboxes of the DES). It acts in parallel on the letters, i.e. as 16 identical 
S-boxes: 
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80,0 80,1 $0,2 $0,3 80,0 80,1 80,2 $0,3 S(s0,0) $(80,1) S(s0,2) $(s0,3), 
1,0 51,1 51,2 51,3 |, 51.0 a ge 18 = S(s1, 0) S(s1, 1) S(s1, 2) S(s1, 3); 
$2,0 $2.1 $2,2 $2,3 $2.9 $2.1 $2,2 $9.3 S(s2,0) S(s2,1) S(s2,2) S(s2,3), 
$3,0 §3,1 $3,2 $3,3 83.9 $3.1 $32 $3.3 S(s3,0) $(3,1) S(s3,2) $(s3,3). 


Let us describe the S-box as a byte transformation: $(G76g--- Go) = 


BrBo vee Bo, where the final result comes from the following affine transfor- 
mation of the space (F2)§ 


Bo 10001111 i 1 
eal 11000111 4 1 
Bo 11100011 ; 0 
63 | |11110001 f 0 
| |11111000|| @ | tT] 0 
a 01111100 a 1 
Bg 00111110 1 
br 00011111 : 0 


The intermediate result 64% --- (6 is computed as follows: 
Bra! + Bea® +--+ + Bla + Bo = (Bra! + Bex? +--+ Bre + Bo)" 


Rose = Fe[z]/(2° + at +2? +241) 
(the byte 00000000 remains fixed). 
The complexity of the S-box is equivalent to the complexity of multiplica- 


tive inversion in Ro56; the philosophy for the ensuing affine transformation is 
merely to create a Boolean transformation without fixed points. 


Exercises 


(1) Compute the inverse transformation of the affine transformation above. 


Solution ; 
10001111 00100101 
11000111 10010010 
11100011 01001001 
11110001 — | 10100100 
11111000 ~ 101010010 
01111100 00101001 
00111110 10010100 


00011111 01001010 
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1.e. 


A 00100101\ (+1 
A 10010010] | +1 
01001001 Bo 
5 |__| 10100100 Bs 
f 01010010]] 4, 
; 00101001] | {44 
6 10010100] | 444 
01001010 3 
ile 


(2) Show that the polynomial description of the affine transformation which 
is the final constituent of the S-box is given by 


Bra” + Bex® + +--+ Bo = [(Bra” + Boa’ + ---+ Boat +a? +2?+241) 
+a5 + 2° + 2 +1] mod (2? + 1). 


(3) The Sbox finally becomes, after some computational effort, the following 
hexadecimal table: 


0}1)/2/)3/4/5/6/7/8]/9}a]lb}/cildle/f 
63|7c|77|7b| f2 |6b} 6£) c5 |30|01)67|2b} fe |d7 |ab| 76 
ca|82|c9|7d| fa|59}47] £0 jad|d4}a2| af |9c}a4|72]}c0 


b7| fd |93| 26) 36] 3f) £7 | cc)34]a5]e5| fl }71}d8]31)15 
04)c7 |23)c3|18]96|05)9a}07) 12/80] e2)eb|27|b2)75 


ba] 78|25 | 2e|1c]a6|b4|c6 | e8 |dd| 74} 1f|4b| bd] 8b] 8a 
70 | 3e |b5| 66 |48} 03] £6 |0e]61|35|57/b9} 86] cl | 1d] 9e 
el | £8 |98|11/69}d9| 8e|94]9b} Le]87}e9 | ce} 55 | 28 | df 
8c |al|89 |Od) bf] e6 |42|68}41]99 | 2d] Of }b0] 54 |bb] 16 


D}Q) 0] To} |] O} CO] NY] OD] OY BY] Co] DO] FE] © 
Ol 
e 
av) 
w 
ms 
oO 
CO 
BS 
oO 
bo 
oO 
a 
ow 
CO 
R 
ion 
ie) 
o> 
a 
Q 
© 
bo 
a 
e 
S 
=o) 
3 
a 
bo 


Read If s;,; = xy, then 5, = 
the position (x,y), 0O<i,7 <3. 
Example §(53) = ed, i.e. $(01010011) = 11101101. 
(a) Verify, following the algorithm defining the S-box, that we indeed get: 
S(88) = c4, i.e. $(10001000) = 11000100. 
(You should avoid computing the multiplicative inverse of x7 + x? modulo 
m() in painfully solving the equation (x? +2°)U+ (a> +a4+a3+24+1)V 
= 1. It is easier to use the table of the 255 powers of € = «+1 in R3,.). 


S'(s;,;) =the letter (8-bit byte) of the table at 


2.2 The Advanced Encryption Standard: The Cipher Rijndael 


(b) Write down the table of the inverse S-box. 


Solution 


0 


52 


7c 


54 


08 


72 


6c 


90 


be 


d0 


3a 


cf 


96 


85 


37 Ie 


AT 


89 


62 


18 


fe 


20 


db 


cO 


cd 


1f 


a8 


31 


12 


10 


80 


60 


Od 


ed 


7a 


93 


c9 


a0 |e 


b0 


c8 


eb 


bb 


83 


53/99/61 


ra] @O | OL} O | OY) DM | CO} CO] NI] GD] OY BY] CW] DO] FE] © 


17 


26 


el 


69 


14 


55 


21 7d 


— 
ye 
wm 


two iterations of the transformation S: 


— 
ol 
<: 


50,0 50,1 50,2 $0,3 
$1,0 $1,1 $1,2 $1,3 
$2,0 $2,1 52,2 $2,3 
$3,0 53,1 $3,2 53,3 


Sos 


i 


§0,0 $0,1 $0,2 $0,3 
$1,0 $1,1 $1,2 $1,3 
$2,0 $2,1 $2,2 $2.3 


§3,0 §3,1 $3,2 $3,3- 


formation which constitutes the second step of the S-box. 
What shall now be the value S(aa) = $(10101010)? 


The ShiftRows Step 


81 


Give an example of a state of four distinct words which is reproduced after 


Modify the S-box of the cipher AES in replacing the multiplicative in- 
version in Rose = F2[2]/(a® + x + x +L + 1) by that in Fo56 = 
Fo[x]/(2§ + 2+ + 23 + x? + 1) — while keeping unchanged the affine trans- 


The first line is kept fixed, the second line is cyclically shifted one byte to the 
left, the following two bytes to the left, and the last line three byte-positions 
to the left. This gives: 


$0,0 50,1 $0,2 50,3 
$1,0 $1,1 $1,2 $1,3 
$2,0 $2,1 52,2 $2,3 
$3,0 53,1 $3,2 53,3 


50,0 $0,1 50,2 $0,3 
$1,1 $1,2 $1,3 $1,0 
$2,2 §2,3 $2,0 $2,1 


$3,3 53,0 53,1 53,2- 
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The MixColumns Step 


Each of the four columns of the state is considered as a vector of length 4 over 
the field Rose. 
Define the circular convolution product of two vectors: 


xo 
Let x = - € R4,¢. The associated circular matrix C(x) is defined by 
2 
v3 
Lo ©3 LQ Ly 
C(a) as TX U3 LQ 
_ XL. L1 Lo L3B 
23 LQ L1 LO 
Example 
02 02 03 01 01 
01 01 02 03 01 
BS Vai |e thet Chek aaa 03 
03 03 01 01 02 


The multiplication of a circular matrix with a vector presents some important 
particularities: Put 


oa) Yo Z0 
XY Y1 ZI 

6 = , = , and z= = C(a2)-y. 
es y 2 . (z)-y 
x3 ¥3 23 


20 = LOYo + U3Y1 + L2Y2 + L1Y3, 
41 = 21Y0 + LOY + U3Y2 1 L2Y3, 
42 = ©2Yo + 2141 T LoY2 T V3Y3, 
23 = L3Y0 + T2Y1 + X1Y2 + LOY3- 


We observe: z; = > 


pu+v=j mod 4 Lu Yv 0 < j < 3. 
Consequences (1) Commutativity: C(x) -y =C(y)-«. 


(2) Invariance under cyclic permutation: C(x) -o(y) = 0(C(«)-y), where 


20 23 
ZA zi 
oO = g 
22 21 
23 v2) 


Combining (1) and (2), we obtain 
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(3) If z = C(a)-y, then C(z) = C(2)C(y) = Cly)C(a2) (multiplication of 
circular matrices). 

It is evident that the properties (1),(2) and (3) generalize for arbitrary n. 


Definition The (circular) convolution product * : R4sg x R4sg —> Rosg: 
Let us define, for 


Ba) Yo 

XY Y1 4 
a |? y yo 256 
v3 Ya 


their (circular) convolution product z=ax*y — by z=C(x)-y=Cl(y)-a. 
We immediately get: 

(1) exy=yre 

(2) o(@*y) = (o(a)) *y = x * (o(y)) 

(3) C(a*y) = C(a)C(y) 


Observation (Polynomial notation) Replace the vectors 


XO Yo 20 

Ty 1 21 
t= > Y= 2 » FSLEYS= 

x2 Y2 22 

x3 ¥3 23 


by the polynomials 


Po(T) = to + 01T + wT? + x3T?, 

Py(T) = yo + iT + yoT? + ysT®, 

p(T) = 2 +27 + 2T? + 237, 

then p.(T) = p,(T)py(T) mod (T* +1), te. Pray(T) is the remainder of divi- 
sion of the polynomial product px(T)py(T) by the polynomial T* + 1. 


Now consider the arithmetical filtering (of the words) of the state array 
given by circular convolution with the “impulse response” 


02 x 
aes Ol) _ 1 

01 1 

03 z+1 


More explicitely, we shall have 


80,0 80,1 §0,2 80,3 80,0 801 80,2 80,3 
$1.0 §1,1 $1,2 $1,3 S19 Sit Sto sis 
$2,0 $2,1 $2,2 $2.3 85.0 854 850 833 
§3,0 §3,1 §3,2 $3.3 $3.9 831 83.0 83.3 
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with 
/ / 
50,0 02 50,0 80,1 02 50,1 
/ / 
,_ | Sio | _ | OL $1.0 Bo Sao s, | OL $11 
Wo = / aa 01 * sy Wy = ! i 01 * ’ 
52.0 $2,0 521 §2,1 
/ / 
$3.0 03 53,0 $3.1 03 $3,1 
cH '02 St 02 
0,2 S0,2 0,3 50,3 
/ / 
' Sy2 | _ | O1 $1.2 1 _ | S13 O01 $1,3 
W2 =, , cae 01 * ’ Ws —_ ! —— 01 * 
$2.2 $2,2 $9.3 82,3 
/ / 
$312. 03 $3,2 $3.3 03 $3.3 
Exercises 


(1) Show that a is invertible for the circular convolution product. You have 


01 
to find b witha*xb=e9 = 00 | 4 
00 
00 
Solution 
Oe et+erte 
f= 09 ze+1 
~ 10d} | a +a741 
Ob e+aetl 
(2) True or false: 
x etart+e 1 
1 ? e+ _ 10 
1 etar+1] 10 
r+1 e+tatl 0 
in (F2[x]/(p(x))* for any irreducible polynomial p(x) over Fo. 
ae 
(3) The vector b as in exercise (1). Then b x . =~ 
ae 
02 x Oe etarte 
Ol] | 1 {| 09] _ ze+1 
es lane, 4 b= | og | =| 840741 
03 r+ Ob x+ae2+l1 


It has been observed that actually b = ax p with a rather simple vector p. 
Consequently, it is possible to implement the decryption operation b * () as a 
composition of a simple preprocessing step p x (), followed by the encryption 
operation a * (). Find p. 


4 C(eo) is the identity matrix, hence eo is the unit for circular convolution. 
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Addition of the Round Key to the State Array 


85 


Here you are in a DES-like situation: You apply 128 times the Boolean oper- 
ation XOR, position by position (bit by bit). Algebraically, one computes the 
sum of two matrices over Rose: 


$0,0 50,1 $0,2 50,3 
$1,0 $1,1 $1,2 51,3 
$2,0 $2,1 $2,2 $2.3 
$3,0 53,1 $3,2 53,3 


ko,0 ko,1 Ko,2 ko,3 
k10 kia 1,2 k1,3 
ko,0 ko ke,2 ke,3 
k3,0 k3,1 k3,2 k3,3 


Exercises 


(1) Find the initial state 


>a 
iy 


YS a 


$0,0 80,1 $0,2 $0,3 

$1,0 $1,1 $1,2 $1,3 

$2,0 $2,1 52,2 $2,3 

$3,0 $3,1 $3,2 53,3 
of a round which guarantees that at the end of the round state = round 
key. 
Give an example of a non-constant initial state (of a round) such that its 
round transform (before the final addition of the round key) is nothing 
but a simple permutation of its 16 letters. 
(Help: Search first for the 8-bit bytes x with S(S(a)) = 2.) 
Consider the following polynomial, with coefficients in Rose: 


SiS ba SEX eo poet OP ee Pa er en eee te Oe 
+09.X7°% + 05X74 


Compute $(00), S(01),.$(02), S(04). 

Let us take for granted:The polynomial S(X) is (as a function on Ro56) 
the S-box of AES-Rijndael. 

Deduce the polynomial T(.X) € R256[X] which is (as a function on Ro56) 
the affine transformation 


Bo 10001111\ / 
pr 11000111 Br 
By 11100011] | & 
Bs 11110001 | | Bs 
Ba 11111000] | 
Bs 01111100] | @ 
By 00111110] | @ 
Br 00011111] \@& 


OoOrroocorF F&F 


(c) How can you finally find the polynomial S~!(X) € Res¢6[X] which is (as 


a function on Ra5¢) the inverse S-box? 
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2.2.3 The Key Schedule 


The round keys will be extracted from the user’s cipher key by means of the 
following algorithm: 


— The cipher key Kk will generate an expanded key array K* which is a vector 
of N words (i.e. a 4 x N matrix of bytes). 
— The length N of the expanded key array K*: N = 4 x (n+ 1), where 


10 AES — 128 
n = Nr =the number of rounds = ¢ 12 AES —192 


14. AES — 256 


— We shall need n+ 1 round keys: Ko, K1,... Kn (n = 10, 12,14), which are 
cut out sequentially from the expanded key array: 


Ko = the first four words of the expanded key, 
ky, = the following four words of the expanded key, 


Ky, = the last four words of the expanded key, 
K* = (Ko, A4,. : glee )s 


But let us begin with the presentation of the algorithm which computes the 
expanded key array from the user’s cipher key. 
First we need two simple operations on words: 


ao S(ao) do a1 

ay S(ay) ra lien a Nee 

S = ag a3 

ag S(a2) a . 

a3 S(a3) and 2 7 

—1 . . 
= : 7 =o ~ for the cyclic permutation o 
co vie introduced together with the circular 
y convolution 
gin} 
: 0 4 : 
RC{i| = 0 € Rosg i> 1. 
0 


(a translation vector, with variable first component). 
Recursive computation of the words wt], 0 <i < 4(n+ 1) 


10 AES — 128, 
n=412 AES —192, 
14 AES — 256, 
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which are the components of the expanded key array. 
w|[O]w[1] --- w[43] if AES — 128 
K* = ¢ w[0)w[l]--- w[51] if AES — 192 


w([O]w{l]---w[59] = if AES — 256 


START: 
4 en AES — 128 


w[0}wil] --- w[Nk—1] is simply the cipher key Nk = | 6 en AES — 192 


8 en AES — 256 


STEP: 
For i > Nk we have wii] = wii — Nk] 6 w|i —1]', where 
: iZ#0mod Nk if Nk = 4,6 
come ne aac if Nk = 8 
wli-1]' = 


S(r(w[i — 1])) @ RC[x_] for i=0 mod Nk 
S(wlt — 1)) for i =4 mod Nk and Nk =8 
END: 71=4n+3. 


Scheme Beginning of the computation of K* in AES-128. 
ko,0 ko,1 ko,2 ko,3 

_ Kio kia k1,2 kis 

~ ka9 ka kee kas 
k3,0 k3.1 k3,2 ks,3 


w([0]w[1]w[2]w[3] = 


S(k13) 01 
w[d] = wld] © ate elo, | 
S(ko,s) 00 
w([5] = w[1] 6 wI4], 
w([6] = w[2] 6 wI5], 
w([7] = w[3] 6 w{6], 
02 
ws] = wid] e 5(r(wlr))) © | |, 
00 
w[9] = w[5] @ wI[8}], 
w([10] = w[6] @ w9I, 
wlll] = w[7] 6 w[10], 
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w([12] = w[8] ® S(7r(w{[11])) @ 


Example of the computation of an expanded key array: The cipher key 
K = 2b Te 15 16 28 ae d2 a6 ab f7 15 88 09 cf 4f 3c. This yields: 


wo = 2b7e1516, wy = 28aed2a6, we = abf71588, w3 = 09cf4f3c. 
index] wl — 1] T() S() |RC[xz]| SRC Juli- Nk] vfi] 
4 | 09cf4f3c | cf4f3c09 |8a84eb01 |01000000) 8b84eb01) 2b7e1516 | a0fafel7 
5 | aOfafel7 28aed2a6 | 88542cb1 
6 | 88542cbl abf71588 | 23a33939 
7 | 23a33939 O9cf4f3c | 2a6c7605 
8 | 2a6c7605 | 6c76052a |50386be5 |02000000]52386be5| a0fafel7 | f2c295f2 
9 | f2c295f2 88542cb1 | 7a96b943 
10 | 7a96b943 23a33939 | 5935807a 
11 |5935807a 2a6c7605 | 7359f67f 
12 | 7359f67f | 59f67f£73 | cb42d28f|04000000) cf42d28f | f2c295f2 |3d80477d 
13 |3d80477d 7a96b943 | 4716fe3e 
14 | 4716fe3e 5935807a | 1e237¢e44 
15 | 1e237e44 7359f67f£ |6d7a883b 
16 |6d7a883b | 7a883b6d | dac4e23c |O8000000] d2c4e23c | 3d80477d | ef44a541 
17 | ef44a541 4716fe8e | a8525b7£ 
18 | a8525b7f 1e237e44 |b671253b 
19 |b671253b 6d7a883b |db0bad00 
20 |db0bad00|0bad00db]2b9563b9] 10000000/3b9563b9| ef44a541 | d4d1c6f8 
21 | d4d1c6f8 a8525b7f | 7c839d87 
22 | 7c839d87 b671253b | caf2b8be 
23 | caf2b8be db0bad00} 11£915be 
24 | 11f915bc | f915be11 |99596582 |20000000|b9596582| d4d1c6f8 | 6d88a37a 
25 |6d88a37a 7c839d87 | 110b3efd 
26 | 110b3efd caf2b8bc | dbf98641 
27 | dbf98641 11f915be | ca0093fd 
28 | ca0093fd | 0093fdca |63dc5474 |40000000 | 23dc5474 | 6d88a37a | 4e54f70e 
29 | 4e54f70e 110b8efd | 5f5fc9f3 
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index} w[i—1] | 70) S(.) |RC[xz]| GRC Jwl- Nk] wii] 
30 | 5f5fc9f3 dbf98641 | 84a64fb2 
31 | 84a64fb2 ca0093fd | 4ea6dc4f 
32 | 4ea6dc4f | a6dc4f4e | 2486842f | 80000000 | a486842f | 4e54f£70e | ead27321 
33 | ead27321 5f5fc9f3 | b58dbad2 
34 |b58dbad2 84a64fb2 | 312bf560 
35 | 312bf560 deabdc4f | 7£8d292f 
36 | 7£8d292f | 8d292f7f£ |5da515d2|1b000000]46a515d2} ead27321 | ac7766f3 
37 | ac7766f3 b58dbad2| 19fadc21 
38 | 19fadc21 312bf560 | 28d12941 
39 |28d12941 7£8d292f | 575c006e 
40 | 575c006e |5c006e57 | 4a639f5b | 36000000) 7c639f5b | ac7766f3 | d014f9a8 
41 |d014f9a8 19fadc21 | c9ee2589 
42 | c9ee2589 28d12941 | e13f0cc8 
43 | e13f0cc8 575c006e | b6630ca6 


The Encryption Protocol 


The encryption with AES—Rijndael is done in 11, 13 or 15 rounds, depending 
on the size of the keys (128, 192 or 256 bits). There are 


— The initial round: Addition of Ko to the initial state (= the plaintext 
block) 


— The rounds n°l1,...n 


O° 


9 


11: as described in (c) 


13 


— The final round: Without the MixColumns step 


Example of an encryption with AES-128: 


e The plaintext block: 32 43 f6 a8 88 5a 30 8d 31 31 98 a2 e0 37 07 34 
e The cipher key: 2b 7e 15 16 28 ae d2 a6 ab f7 15 88 09 cf 4f 3c 
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round initial after after after round 
state SO - a * () key 

32]88]/31/e0 2b/28}ab]09 

0 43/5a]31|37  |7elae| f7 | cf} — 
£6 | 30/98/07 15|d2/15| 4f 
a8|8d|a2/34 16] a6] 88] 3c 
19]|a0]9a]e9 d4]e0|b8]|le| |d4]e0|b8}1le 04] e0}48/28 a0|88/23}/2a 

1 3d|f4|[c6] f8 27) bf|b4]41 bf |b4/41]27 66|/cb]| f8|06] gq | fa|54/a3]6c] — 
e3}e2/8d]48 11]98]5d]52 5d]52/11/]98 81)19}d3/26 fe | 2c |39|76 
be] 2b] 2a]08 ae]f1]e5/30] |30]ae] fl ]e5 e5|9a]7al 4c 17/b1/39]05 
a4/68|6b|02]|49/45/ 7f/77]|49145] 7£177||58]1b|/db]1b f2/7a|l59|73 

2 9c| 9f |5b/6a of |db/39]02] |db/39]|02] of | |4d/4b]e7/6b] qm |[c2]96]/35]/59] _ 
7£|35/eal50| |d2/96]87]53 87|53/d2/96]}ca}5a}]ca|bO 95|b9|80| f6 
f2|/2b/ 43/49] /89] f1 |la/3b]/3b/]89|fl]la fl }ac|a8]e5 £2 /43]7al| 7£ 
aa|61]82/68 ac| ef |13]45 ac] ef |13]45 75|20/53]bb 3d]47|/1le|6d 

3 8f|dd|/d2/32 73|c1/b5] 23 cl |b5|23)73 ec|Ob]cO] 25] q |80]/16]23] 7a} — 
5f|e3 | 4a] 46 cf ]11]d6]5a]/d6]5a] cf {11 09163] cf | do 47|fe|7e|88 
03] ef |d2]9a||7b|df]b5/b8]|/b8]7b] df] b5 93/33] 7c] dc 7d|3e|44]3b 
48|67|4d|/d6 52]85]/e3]| f6 52/85]e3]f6 Of | 60] 6f | 5e ef |a8|[b6|db 

4 6c|1d]e3 | 5f 50|a4]11] cf a4/11]| cf |50 d6|31}c0O/b3} q |44|52]71]0b] — 
4e|9d|/b1]58 2f|/5e]c8]6a c8]6al 2f[5e da|38/10/13 a5|/5b/[25]ad 
ee |0d| 38 | e7 28/d7/07/94 94|28]d7]07 aQ9|bf|]6b/01 41] 7f£|3b]00 
e0|c8]d9]85 el |e8 | 35/97 el]e8|]35|97] |25|/bd|b6] 4c d4]7c}]ca]il 

5 92/63}b1]b8 4f|fb|c8|6c fb}]c8|[6c|4f} }d1]}11]3al4c] q |d1] 83] f2}]f9 | — 
7f£|63]35|/be d2|fb|/96/ae 96] ae|d2| fb a9}d1/33]cO c6|9d]}]b8]15 
e8/c0}50/01 9b} bal 53] 7c 7c|9b|ba| 53] |ad] 68 | 8e|}bO £8187|bc|be 
fl |cl|7c]5d al|78/10]4c al|78/10]4c Ab | 2c|33|37 6d]11}db]ca 

6 00/92} c8]b5 63] 4f|e8]d5 4f |e8}d5|63 86|4a]9d|/d2] q |] 88]0b| £9 ]OO} _ 
6f | 4c]8b/d5] |d8]29/3d] 03 3d|03]a8/29 8d/89| f4]/18 a3|3e/ 86/93 
55] ef |32]0c fc | df | 23] fe fe | fc | df |23 6d|80]e8/d8 7alfd|41|fd 
26/3d]e8 | fd £7|27|9b|54 £7|27/9b| 54 14|46] 27/34 4e| 5f|84|4e 

ré Oe] 41|64]d2]| |ab/]83|43]b5] [83/43] b5/ab 15/16]46]2a] q [54] 5f]a6/a6] —_ 
2e|b7|72|8b 31]a9| 40] 3d] |40]3d]31]a9 b5]15/56/d8 £7|c9| 4f }dc 
17|7d|a9]25 f0 | ff |d3] 3f 3f|f0] ff [d3 bf |ec|]d7| 43 Oe| £3 [b2| 4f 
5a]19]a3]7al [beld4]0a]da][be]d4/Oa]da][00]b1]54] fa ea]b5|31] 7f 

8 41]49]e0/8c 83|/3b]e1|/64] |3b]el1/64] 83 51]c8|76)1b] q |d2/8d/ 2b] 8d} — 
42|/dc|]19/04 2c]86]d4] f2 d4] f2 | 2c]86 2f/89]6d]99 73]/bal #5129 
b1]1f]65] 0c c8|c0]4d] fe fe |}c8|cO|4d]/d1] ff |cd]ea 21/)d2|60| 2f 
ea|04/65|85 87|f2]4d|97 87|f£2|4d]97 47|40]a3] 4c ac|19|28]57 

9 83/4515d/96 ec |6e]4c}90 Ge | 4c | 90] ec 37)d4|70| 9f| @ 177) faldi[5c} — 
5c|]33]98]b0| |4a]c3]46]e7 46/e7|4a]c3 94|e4]/3a] 42 66|dc}]29]00 
£0 |2dJ/ad|c5 8c|d8]95]a6 a6 |8c}d8}95 ed|a5]/a6/bc £3 /21/41|6e 
eb|59/8b/1b e9|cb|3d| af e9 |cb|/3d] af d0|c9/el|b6 

10 40|2e}al}]c3 09] 31]32]2e 31/32] 2e/09 @ |14]ee | 3f] 63] — 
f2/38/13]42 89|07] 7d] 2c 7d|2c|89|07 £9 |}25]/0c| Oc 
le|84]e7]/d2 72/5f/94]b5 b5/72[ 5f]94 a8|89/c8|a6 
39]/02}dc]19 
25|dc}/11]/6a 
84]/09|85]0b 
1d] fb [97/32 


e The ciphertext: 39 25 84 1d 02 dc 09 fb dc 11 85 97 19 6a Ob 32 
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Exercises 


(1) The plaintext unit: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00. 
The cipher key: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00. 
The state array at the beginning of round n° 2? 

(2) Your cipher key: 43 c6 84 53 33 25 Oc 80 1d 2b c3 97 e2 cc 40 b3. 
Your plaintext block: 4a a3 3c 69 4f 4f 3b ad 59 7f £3 d9 ec e8 32 Oc. 
The state array at the beginning of the second round? 

(3) In the example of encryption with AES-128 (on the preceding pages) the 
third round transformation transforms the state array 


aa 61 82 68 48 67 4d d6 
pes 8f dd d2 32 eae ae 6c ld e3 5f 
5f e3 4a 46 de 9d b1 58 
03 ef d2 9a ee Od 38 e7 
A modification of four bits in the initial state I gives 
11 67 4d d6 
re 4b 1d e3 5f 
ed 9d bl 58 
fd Od 38 e7 


Find the error. 


O1 | 04 | 85 | 88 


91 | 04 | 46 | 44 


(4) The eighth round key: Kg = 81 bod 1 a7 1 28 


cO | Oc | 18 |} 11 


The ninth round key: Kg =? 

(5) What was the initial state of a (standard) round that produces as the 
result of the third step — i.e. before the addition of the round key — the 
following state array: 


7c | 77 | 7b | f2 


7c | 77 | 7b | f2 


7c | 77 | 7b | f2 


7c | 77 | 7b | f2 


(6) The plaintext block: 00 11 22 33 44 55 66 77 88 99 aa bb cc dd ee ff. 


The cipher key: 00 01 02 03 04 05 06 07 08 09 0a Ob Oc Od Oe Of. 
The initial state of the second round will be 

89 d8 10 e8 ** ** ** ** 2d 18 43 d8 cb 12 8Ff e4. 
Compute the four masked bytes. 
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2.2.4 Decryption with Rijndael 


First consider the decryption algorithm in its logical structure: We have simply 
to apply the inverse transformations of the encryption algorithm, with the 
order of the transformations inverted. 

This will give, for a standard round of inverse AES-Rijndael: 


(1) The addition of the round key 
(2) The inverse convolution on the words of the state array 
(3) The inverse “rotation” of the lines of the state array 
(4) The inverse S-box. 

But the cipher Rijndael seems to have been designed with a tender look 
towards the old DES, where the decryption is nothing but an encryption of the 
ciphertext (with the order of the round keys inverted). Here, the situation is 
less symmetrical; but altogether, it is possible to create something similar: the 
decryption operations may be executed in the order of encryption, in replacing 
only each constituent transformation by its inverse. The order of the round 
keys will be inverted, but the round keys will be “inverted” themselves. 

To better understand this, consider a simplified version of AES: 


Example AES—Rijndael inverse in two rounds: 


(1) Addition of the round key K2 

(2) (Inversion of the) rotation of the lines of the state array 

(3) Operation of the inverse S-box 

(4) Addition of the round key Kk, 

(5) Inverse convolution operation on the words of the state array 
(6) (Inversion of the) rotation of the lines of the state array 

(7) Operation of the inverse S-box 

(8) Addition of the round key Ko 


Observation (1) The operation of the S:box commutes with the operation 
of the line rotations (since the first simply acts on the letters, in parallel). 
(2) The composition (addition of the round key Kj) with (inverse convolution 
operation on the columns of the state) 
equals the composition (inverse convolution operation on the columns of 
the state) with (addition of the inverted round key K*) where K? = b « K; 
(column by column). 


(this is an immediate consequence of the linearity of the operation of circular 
convolution 


Oe et+a2+ea 
{09 ] | z+] 
Recall: b= al amd ee 


0b e+tatl 
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Consequences The decryption by AES-—Rijndael in two rounds, in 
encryption-adapted order: 


( 

(2) Operation of the inverse S-box 

(3) (Inversion of the) rotation of the lines of the state array 

(4) Inverse convolution operation on the words of the state array 
(5) Addition of the round key K# 

(6) Operation of the inverse S-box 

(7) (Inversion of the) rotation of the lines of the state array 

(8 


The structure initial round — normal round — final round has been respected. 
The order of the operations corresponds to their order in the encryption al- 
gorithm — up to the inversion of the particular transformations. 

This will clearly also be true in the general case. 


The round keys iv } for the decryption algorithm are obtained as follows: 


(1) Compute the expanded key array K* = (Ko, Ki,..., Kn), n= < 12 
(2) Let Nr (= 10, 12, 14) be the number of rounds. then 


KH eK K; fori =0,Nr 
v bx K; otherwise 


The round keys Ki will operate in inverse order: aoe Tesi byt ikee, Ki. 


2.3 The Public Key Paradigm and the Cryptosystem 
RSA 


The idea of a public key system dates from 1976, and is due to Diffie and 
Hellman. The first realization of such a system is the cipher RS'A, published 
in 1978 by Rivest, Shamir and Adleman. 

One observes that almost all technical ideas around public key ciphers refer 
to encryption methods by exponentiation (in certain multiplicative groups 
supporting the encoded information). 


2.3.1 Encryption and Decryption via Exponentiation 
Still more Arithmetic 


Let us introduce first Fuler’s totient function y(n): 
y(n) = the number of integers a between 1 and n which are prime to n. 
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Example 
y(2)=1, (8) =4, 
y(3)=2, (9) =6, 
(4) = 2, (10) = 4, 
(5) = 4, (11) = 10, 
p(6) = 2, y(12) = 4, 
p(7) =6, y(13) = 12. 


Important observation. y(n) counts the number of residues which are 
invertible for the multiplication in Z/nZ (Recall: [a] is invertible in 
Z/[nL <> gcd(a,n) = 1). This means: y(n) is the order of the mul- 
tiplicative group(Z/nZ)*. 


Computation of ¢(n): 
(1) y(p*) = p*41(p- 1), if p is prime, k > 1. 
(2) y(m-n) = v(m): y(n), if m and n are coprime. 


In particular: y(p-q) = (p—1)(q—1) for two distinct prime numbers p and gq. 


Let us insist: y(n) is calculable if one knows the prime factorization of n. 
Example (792) = y(23) - y(37) - (11) = 4- 6-10 = 240. 


It is surprising that the mathematical basis for most of the arithmetical 
ciphers is a very, very elementary result. It is the 


Theorem (Lagrange) Let G be a finite (multiplicative) group of order y. 
Then everya€G satisfies a? = 1. 


Proof (for commutative groups) Choose your a (arbitrary, but fixed), and 
an indexing of the group elements: x1, %2,...,%,. Multiply all of them by 
a: {a%1,a%2,...,aty,} = {@1,%2,...,2,}. Hence: (ax1)(axe)--- (ary) = 
1%Q+++XLy. This yields a? = 1, as claimed. 


Consequence (Euler) a?) = 1 mod n, whenever a is prime to n. 


In particular (Fermat): a? = a mod p for every a € Z and every prime number 
D. 


The situation in which we are particularly interested is the following: 


Let G be a multiplicative (abelian) group of y elements (which encode the 
blocks of our information). We aim at encrypting (decrypting) via expo- 
nentiation in this group. How can the exponents e of encryption and d of 
decryption be harmonized? 
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The answer is simple and basic for all arithmetical cryptography: 


Let y be the number of elements of G. 
Suppose that ed=1mody 


Then: (a?)* =a for every a € G. 


The argument: Write ed=1+k-y. Then (a°\e =a: (a?) oe 
since a®=1 (Lagrange - Euler). 


Systems of Encryption by Exponentiation 


At this point, our first idea should be the following: 
Let us digitize our message blocks (in the broad sense) as remainders of 
division in “natural” multiplicative groups of the type 


(1) G = (Z/pZ)* = the group of the p— 1 non-zero residues of the field Z/pZ, 
p prime. 

(2) G = (F.2[z]/(p(x)))* = the group of the 2" — 1 non-zero residues of the 
field F2[x]/(p(x)), p(w) = a” +---+1 irreducible over Fp = {0,1}. 


Note that at first approach we have counted on cyclicity: The relevant groups 
are all generated by a single element (which is not unique): We consider dis- 
crete circles. 

The arithmetical cryptography encounters a typical situation here: When- 
ever these groups are sufficiently big, then the arithmetic will be blind in face 
of geometry; the cyclicity will be algorithmically inaccessible (the problem of 
the discrete logarithm will be difficult). 

On the other hand, if the “discrete circle” is algorithmically well con- 
trolled (cf. the log tables for Fo5¢ and for R56, serving as arithmetical support 
for AES), then the multiplicative structure of G will be trivialized down to 
an additive structure, by the discrete logarithms (the “angles” replacing the 
“points” ), and the cryptographic interest of G will vanish. We shall make all 
this more precise in a moment. 

But let us return to the groups G that we proposed above. The relation 
between the encryption exponents e and the decryption exponents d is then 
the following: 


(1) If ed=1mod (p—1) then (a°)? = a mod p. 

(2) If ed=1 mod (2” — 1) then (r(x)°)4 = r(x) mod p(z). 

We shall call the cryptosystems based on exponentiation in the groups (Z/pZ)* 
or (F2[2]/(p(x)))* uniformly Pohlig—Hellman systems. 
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Exercises 


First a technical complement: 
Fast exponentiation. 


e Fundamental idea: Compute a° mod n in such a way that the arithmetic 
never exceeds the size of n? (compute r(x)° mod p(x) in such a way that 
the arithmetic never exceeds 2 times the degree of p(x)). 

e Realization of the idea: Write the exponent e in binary notation, i.e. as a 
sum of powers of 2; the exponentiation will then essentially reduce to the 
(iterated) computation of squares. 


Example 35091" mod 4717 =? 
[3509]1” = [3509}'®[3509] 


3509]? = [1711] 
35094 = [1711]? = [2981] 


3509]® = [2981]? = [4250] 


3509]!6 = [4250]? = [1107] 
Hence: 3509!” mod 4717 = 2372. 


(1) A Pohlig—Hellman system, in binary polynomial arithmetic modulo p() = 
x*+2+1. Your encryption exponent: e = 13. 
Compute the ciphertext C corresponding to M = 0101 = x? +1, then d, 
your decryption exponent, and verify, in decrypting, the restoration of M. 
(2) A Pohlig—Hellman system, in binary polynomial arithmetic modulo p(x) = 
a8 +a4+4@°+27+1. Your encryption exponent: e = 32. You receive the 
ciphertext C = 10011101. Decrypt! 
(3) The same system as in exercise (2). 
(a) A encrypts the plaintext M = 11010101 into the ciphertext C = 
01000011. 
Find her encryption key e and her decryption key d. 
(b) The same question for B, who encrypts the plaintext M = 11010010 
into the ciphertext C = 11001001. 
(This exercise is a first illustration to our remark on the trivialization of 
an arithmetical cryptosystem in case the problem of the discrete logarithm 
is easy. Here, we control perfectly the discrete circle of the 255 non-zero 
residues modulo p(x) = «8 + 24 + 2° + x +1). 
(4) Still the same system. The user F' tells you that he encrypted the plaintext 
11111111 into the ciphertext 10101010. You find, he is a practical joker. 
Why? 
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2.3.2 The Cryptosystem RSA 
The Algorithm RSA 


The Pohlig—Hellman system is symmetrical, exactly like the ciphers DES and 
AES: The knowledge of the keys for encryption is equivalent to the knowledge 
of the keys for decryption. Let us make this more precise: 

Let (e€4,pa) be the encryption key of A (A encrypts her plaintext blocks, 
digitalized in remainders of division modulo p,4 — a big prime number — via 
exponentiation with e, in Z/p,Z). Then her decryption exponent d, is simply 
the solution (in X) of the equation e4X + (p4 —1)Y =1. 

RSA creates an asymmetrical situation with an arithmetical triteness: One 
replaces the prime number p, which defines the arithmetic of the user A by a 
composite number n4 = paqga, where p, and qa are two (big) distinct prime 
numbers. 

The exponents of encryption e, and of decryption d, are connected by 


eada =1 mod y(na), (na) = (pa —1)(¢a —- 1) 


(Recall: It is the theorem of Euler relative to G4 = (Z/n4Z)* which controls 
the relation between the exponents e, and da). 


Remark As soon as you have found and attributed e4 and d,4, you are allowed 
to encrypt and to decrypt in all of Z/n4Z: We will have (a°4)44 =a mod na 
for every a, invertible for the multiplication modulo n, or not. This is an 
immediate consequence of the Chinese Remainder Theorem (Exercise). 

We note: We succeeded in complicating the Pohlig—Hellman situation. But 
what are now our benefits? 

The answer: 


The advantage of the new situation is the possibility of making public 
the encryption keys whilst keeping secret the decryption keys. 


The Functioning of the Cipher RSA 


Every user A of the cryptosystem RSA enciphers and deciphers in its individ- 
ual arithmetic, i.e. it treats its plaintext (ciphertext) blocks as remainders of 
division modulo n4, where n4 = paqa is a product of two big distinct prime 
numbers. 

The exponents of encryption e,4 and of decryption d, are multiplicative 
inverses modulo y(n4) = (pa — 1)(qa — 1). 

The encryption keys (e4,n4) are public (in some directory). 

The decryption keys (d4,n,) are private (secret). 


Problem How is it possible not to reveal the decryption exponents, while 
rendering public the encryption keys? 
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Let us try to compute d, in function of (e4,n,). 

We know: dy is the solution (in X) of the linear equation e4X+y(n4)Y = 
1. da is calculable under the assumption that (na) will be calculable in 
function of n4. But this is equivalent to finding the two prime factors p, and 
dA of NA- 

But the factorization algorithms for big integers are slow. 

If the size of ny is sufficiently large, it is practically impossible to find (by 
algorithm) the two prime factors p, and qa. 

The historical challenge came from the authors of the system RSA them- 
selves, who proposed, in 1978, to all of the planet earth the factorization of an 
integer of 129 decimal positions (the integer RSA-129). Seventeen years later, 
in 1994, Atkins, Graff, Lenstra and Leyland succeeded, with the aid of 1,600 
computers connected via Internet, in finding the two prime factors (of 64 and 
65 decimal positions). 

Today the size of 1,024 binary positions (for the n4) is considered to 
be sufficient to guarantee the security of the system RSA. Note that the 
key attribution demands the generation of big prime numbers py, and qu, of 
512 binary positions each. This has created an algorithmic brotherhood of 
pseudorandom generators (in order to produce the candidates) and of good 
primality tests (in order to choose the winners). We shall treat this subject in 
a moment. 


Attention e Conceptually, the keys of the system RSA are attributed for 
a relatively long period (cf the RSA-129 challenge above). 

e Practically, there is an important exception: When using the algorithm 
RSA for digital signatures (in the standard rDSA), the encryption expo- 
nent e may even be shared by an entire group of users, whereas the private 
keys (da,na) will change with every new signature. 


Finally, let us point out that RSA is slow. For example: DES is at least 
1,000 times faster than RSA. 

As a consequence, RSA will mainly be used for “little confidences” — like, 
for example, the transmission of the cipher key of a huge document encrypted 
with AES. 


RSA as a Public Key System 


We shall denote by Fx (-) and Dx(-) the encryption and decryption transfor- 
mations of the user X. 


Personal Envelopes 


The user A would like to send a (secret) message M to the user B. 

She encrypts M +> C = Ep(M)_ with the public encryption transfor- 
mation of the receiver B (she shuts the envelope for B). Only B can open: He 
recovers the plaintext using his private transformation: C > M = Dp(C). 
The confidentiality is thus guaranteed. 
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Signatures Inside Personal Envelopes 


In a first approach, we shall adopt a simplifying and somehow academic view- 
point, which reflects nevertheless exactly how the functioning of RSA has been 
perceived during the first 15 years of its existence. 

The sender A reserves a particular patch P of her message M for the 
signature (P would be, nowadays, the “message digest” of the full plaintext — 
we shall treat the standardized digital signatures in Sect. 2.4). The receiver B 
should then be able to verify whether the integrity has been respected. How 
shall we proceed? 

The logic of the affair will not change if we suppose that P = M. 


(i) A signs in treating M as a ciphertext: she computes D,4(M) with her 
private key, 
(ii) Then she sends C = Eg(Da(M)) to the receiver B. 
(iii) B opens the envelope, in decrypting with his private key; he obtains 
Dp(Ep(Da(M))) = Da(M). 
(iv) Finally, B verifies the signature of A: he applies the public transformation 
of the sender and gets correctly M = E4(Da(M)). 


Attention The procedure described above only works for na < ng ! (why?) 
But in the opposite case (and this will be a public relation!) we may pass to 
an inversion: A will send to B the ciphertert C’ = Da(Ep(M)) and B will 
first apply E4 and then Dg. 


Example of a simple encryption: 
yp(na) = 52-60 = 3120. 
ea =71 dy =791. 


M = RENAISSANCE 


Let us encode in decimal 4-digit blocks (of two letters) according to 
A= 00, 


B=0l, 


Z = 25, 


space = 26, 


100 
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M=RENAIS SANC E- =170413000818180013020426, 
(e4,na) = (71,3238), 

17047! = 3106 mod 3233 (Fast exponentiation!), etc. 

We obtain finally: C = E.4(M) = 310601000931269119842927. 


Exercises 


(1) 


— 
w 
wm 


— 
ye 
Nae 


We consider a mini-RSA, with plaintext (and ciphertext) units consisting 
of decimal 4-digit blocks (i.e. of two letter blocks: A = 00,B =01,...,Z7= 
25, space = 26). 

The user B receives the ciphertext C' = 2433 which is a signed message 
of the user A. 

The public keys: 


(ea, 4) = (53, 2923) 
(ep, ng) = (233, 4087) 


Decrypt the message. 
The same situation as above. 
The user B receives the ciphertext C = 1441 which is a signed message 
of the user A. 
(ea, 4) = (11, 2867) 


The public keys: (ep, np) = (689, 2987) Decrypt the message. 


Let FE: (Z/nZ)* —+ (Z/nZ)*,x+> 2* be the operation of encryption 
in RSA. 

Show: There exists m > 1 such that E(x) = x for every x € (Z/nZ)*. 
E™ = m-times iteration of the encryption operation. The plaintext can 
(surprisingly) be recovered from the ciphertext by iterated encryption! 
(Help: Use the theorem of Euler, “on the right level”.) 

Still another mini-RSA, with encryption and decryption in decimal 4-digit 
blocks (i.e. in two letter blocks: A = 10, B = 11, C = 12,...,Z = 35, 
espace = 99). The user B receives the ciphertext C = QNSE, which is a 
signed message of the user A. 

The public keys: (e4,n4) = (87,4331), (eg,ne) = (1493,5561). 
Decrypt! 

The situation of the exercise (4). 

The user B receives the ciphertext C = 0141.2685, which is a signed 
message of the user A. 

The public keys: (e4,n4) = (37,5183), (ep,ngB) = (2491, 5293). Decrypt! 
Once more the situation of exercise (4). The user B receives the ciphertext 
C = 2176.3509, which is a signed message of the user A. 

The public keys: 

(ea, Ma) = (17, 4559), (ep, np) = (2961, 4717). Decrypt! 
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(7) Still the situation of exercise (4). The user B receives the ciphertext C= 
3997.3141, which is a signed message of the user A. The public keys: 
(e4,na) = (67,4307), (ep, ng) = (3973, 4331). Decrypt! 


2.4 Digital Signatures 


The algorithms of signature are standardized® in Digital Signature Standard 
(DSS), the last version of which dates from January 2000. The standard is 
given in three variants: the DSA, which is the principal signature algorithm, 
the rDSA, which precises the use of the cipher RSA for digital signatures, 
and the ECDSA, which is the most fashionable signature algorithm, using 
triumphantly the arithmetic of elliptic curves. We shall present all three algo- 
rithms. But, to begin appropriately, we have first to speak of the “support” 
for the signatures, i.e. we have to speak of message digests. 


2.4.1 Message Digests via SHA-1 


The algorithm Secure Hash Algorithm (SHA-1) creates a message digest of 
160 bits of a message (of a data file) of any length less than 2% bits. 

The security of the algorithm is understood as follows: it is computa- 
tionally infeasible to create a message which will be transformed into a given 
digest. Similarly, is it computationally infeasible to find two different messages 
with the same message digest. 

A modification of the message text during its transmission will give — with 
a very high probability — a new message digest that is different from the digest 
which has been extracted by the sender. If the signatory of the message and 
the verifier of the signature use both the message digest of the document as the 
“algorithmic support” of the signature, then the signature will be “attached 
to the document”. 

The algorithm SHA-1 is modeled after the MD4 (Message Digest Algo- 
rithm) of Ronald L. Rivest (1990). 

SHA-1 became a standard in 1995. 


Message Padding 


Language: A word will be a binary string of 32 bits; we shall use hexadecimal 
notation almost exclusively: 

1010 0001 0000 0011 1111 1110 0010 0011 =al03fe23. 
A block will be a binary string of 512 bits, i.e. a block consists of 16 words. Now 
let M be the message, considered as a binary string of length | (J < 26+). The 
purpose of message padding is to make the total length of a padded message a 


5 At least for the United States... 
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multiple of 512. The “blowing-up” of M into the padded message M* — which 
will be a sequence of n blocks: 

M* = M{1M[2]---M[n], n>1. 

The length /* of M* thus becomes a multiple of 512. The construction of 
M* is simple: M* = M10---0w we. The (long) prefix is M, followed by 
the bit 1, then by a certain number of zeros, finally by 64 bits wyw2 which 
are the binary notation (over 64 bits) of the length J of M. The number of 
intermediate zeros that are appended will be minimal for the length of M* to 
be a multiple of 512. 

In computing the message digest of M, the algorithm SHA-1 will treat, 
one by one, the blocks M[1], M[2], ..., M[n] of M*. 


Example M =01100001 01100010 01100011 01100100 01100101 

the length is 1 = 40; this yields for M*, written as a block of 16 words (in 
hexadecimal notation): 

M* = 61626364 65800000 00000000 00000000 00000000 00000000 00000000 
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 
00000028. 


The Algorithm SHA-1, Global Structure 


SHA-1 computes the message digest m = SHA W— 1(M) in function of M* = 
M{1]M[2]---M[n]. We shall have n exterior rounds, one for (the treatment 
of) each block of M*. Every exterior round consists of 80 interior rounds. 
The algorithm is (structurally) best understood if you look at it this way: 
The message digest m = SHA — 1(M) is the ciphertext of a fixed plaintext 


Mo = 67452301e fcdab8998badc f e10325476c3d2e1 f0 


(which is the same for every run of the algorithm). M enters into the algorithm 
via M* = M[1|M{2]---M[n] which is a cipher key vector. Every M{i] is 
blown up into an expanded key array ( for the i-th exterior round) of 80 words 
WoW,---W7o ( recall the key schedule of AES—Rijndael). Each of these 80 
words (of 32 bits) will serve as a round key for an interior round. 
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Scheme of an exterior round (treatment of a block M[i]): 


H= Ao| Ai |H2|H3| Aa, 160 bits 
1 
t=0,1,..., 79 80 interior rounds 
Xt = A|BIC|DIE, 160 bits 
| — Wi, Ky 


T =T(Xt, Wi, Kt) 


1 
Xt41 = T|A|B?/C|D 
1 
1 
ise Ho + Al\A, + B)H.o+ClH34+ Di\H,+E 160 bits 


Let’s sum up: 


The algorithm SHA-1 computes the message digest m = SHA-1(M) using 
the padded version M* = M[1]M[2]---M[n] of M to transform a universal 
plaintext mo (160 bits) into the ciphertext m (160 bits). 

The Mt] serve as keys for “exterior rounds”. A key schedule generates, 
for each Mi], 80 round keys WoW, ---W79 for 80 “interior rounds”. 

We remark the presence of two five-word buffers: 


(i) HoH, H2H3H, which will be the exterior round state array. 
(ii) ABCDE which will be the interior round state array. 


A buffer T of 32 bits is to be added; its content corresponds to f(Rj-1, Ki) 
in DES. The content of T changes with every interior round. 
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The Algorithm SHA-1 in Detail 
(1) Initialization of the Buffer H 


Ay = 67452301, Hz = 98badcfe, Ay = 03d2e1f0, 
A, = efcdab89, H3 = 10325476, 


(2) Two Operations on Words: 


The algorithm SHA-1 uses several operations on (binary, length 32) words; 
most of them are rather familiar. Two of them should be, however, underlined: 


(i) The cyclic left shift: S(x122 eee X32) = 12X%3°°* X39%1 
(ii) The sum: Z = X + Y = >| Z |=| X | +| Y | mod2*? 


(where | . | means “numerical value of the binary word”. Example: 
| c3d2e1f0 | = 3285377520) 


Sequential treatment of M* = M[1])M{2]---M[n): 
fori =0,1,...,n: 
(3) Expansion of M{i] into WoW, ---W79: 


M{i] = WoW, --- Wis. 


Recursive computation of the word W;, 16<t< 79: 


W, = S(Wi_-3 ® Wi-g © Wi-14 © Wi-16). 


(4) Initialization of the Interior Round Buffer: 


A= Ho, C = Ap, E= Hg, 
B= AHA, D= Hs, 


(5) Processing of an Interior Round: 


t=0,1,...,79: 
T=S°(A)+fi(B,C,D)+E+Wi+ Ki, 
where 
(BA C)V ((=B) AD) 0<t<19, 
B@CeD 20 <t< 39, 
f:( B,C, D) = 


(BAC)V(BAD)V(CAD) 40<t<59, 


BéeCEeD 60 <t< 79. 
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Attention The Boolean operations on the words are executed bit by bit (i.e. 
32 times in parallel). 


5a827999 O0<t<19, 

6ed9ebal 20 <t < 39, 
Ki= 

8flbbcde 40<t< 59, 


ca62cld6 60<t< 79. 


Then A=T, B=A, C=S°9(B), D=C, E=D, (We note the formal 
similarity of an interior round with a round of a “5-word generalized DES”). 


(6) End of the Exterior Round: 


Ho = Ho +A, 
A, =A, +B, 
Hy = A2+C, 
Hf, = 43+ D, 
A, =H,z+E#. 


Example Consider the text M= “abcdbcdecdefdefgefghfghighijhijkijkljklm- 
klmnlmnomnopnopq” in ASCII. 

This text of 56 8-bit characters has length | = 448. The hexadecimal 
representation of 448 (over 64 bits) is 00000000 000001c0. We will have M* = 
M(1)M [2] = M100---000000000 000001c0. 

Sa —” 

Let us begin with the first exterior round. The words of the first block are 
0] = 61626364, 
1] = 62636465, 
2| = 63646466, 
3] = 64656667, 
4] = 65666768, 
5| = 66676869, 
6| = 6768696a, 
7| = 68696a6b, 
8] = 696a6b6c, 
9| = 6ab6b6c6d, 
1 
1 
1 
1 


0] = 6b6c6d6e, 
1] = 6c6d6e6f, 
2| = 6d6e6£70, 
3] = 6e6f7071, 


2A SAS SSS SS SS 
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W[14] = 80000000, 
W([15] = 00000000, 


The values for A, B,C, D, E after the tth interior round are the following: 


A B C D E 
=0: 0116fc17 67452301 7bf386ae2 YI8badcfe 10325476 
t=1: ebf3b452 0116fcl17 59d148c0 7bf36ae2 Y98badcfe 
t=2: 5109913a ebf3b452 cO045bf05 59d148c0 7bf36ae2 
t=3: 2c4f6eac 5109913a bafced14 c045bf05 59d148c0 
=4: 33fdae5b 2c4f6eac 9442644e bafced14 c045bf05 
t=5: 96b85189 33f4ae5b Obl3dbab 9442644e bafced14 
t=6: db04cb58 96b85189 ccfd2b96 Obl3dbab 9442644e 


t=7: 45833f0f db04cb58 65ae1462 ccfd2b96 Ob13dbab 
t=8: c565c35e 45833f0f 36c132d6 65ae1462 ccfd2b96 
t=9: 6350afda c565c35e d160cfc3 36c132d6 65ae1462 


t=10: 8993ea77 6350afda b15970d7 d160cfc3 36c132d6 
t=11: el9ecaa2 8993ea77 98d42bf6 b15970d7 d160cfc3 
t= 12: 860348le el9ecaa2 e264fa9d 98d42bf6 b15970d7 
t=13: 32f94a85 860348le b867b2a8 e264fa9d 98d42bf6 
t=14: b2e7a8be 32f94a85 al80d207 b867b2a8 e264fa9d 
t= 15: 42637e39 b2e7a8be 4cbe52al al80d207 b867b2a8 
t=16: 6b068048 42637e39 acb9ea2f 4cbed52al al80d207 
t=17: 426b9c35 6b068048 5098df8e acb9ea2f 4cbe52al 
t=18: 944blbd1 426b9c35 lacla012 5098df8e acb9ea2f 
t=19: 6c445652 944blbd1 509ae70d lacla012 5098df8e 
t = 20: 95836da5 6c445652 6512c6f4 509ae70d lacla012 
t= 21: 09511177 95836da5 9b111594 6512c6f4 509ae70d 
t = 22: e2b92dc4 09511177 6560db69 9b111594 6512c6f4 
t = 23: £d224575 e2b92dc4 c254445d 6560db69 9b111594 
t = 24: eeb82d9a £d224575 38ae4b71 c254445d 6560db69 
t= 25: 5al42cla eeb82d9a 7f48915d 38ae4b71 c254445d 
t = 26: 2972f7c7 5al42cla bbaeOb66 7f48915d 38ae4b71 
t = 27: d526a644 2972f7c7 96850b06 bbaeO0b66 7f48915d 
t = 28: e1122421 d526a644 cadcbdfl 96850b06 bbae0b66 
t = 29: 05b457b2 ©1122421 3549a991 cadcbdfl bbae0b66 
t = 30: a9c84bec 05b457b2 78448908 3549a991 cadcbdfl 
t= 31: 52e31f60 a9c84bec 816dl5ec 78448908 3549a991 
t = 32: 5af3242c 52e31f60 2a7212fb 816dl5ec 78448908 
t = 33: 31c756a9 5af3242c 14b8c7d8 2a7212fb 816d15ec 
t = 34: eYac987c 31c756a9 16bcc90b 14b8c7d8 2a7212fb 
t = 35: ab7c32ee e9ac987c 4c7ld5aa 16bcc90b 14b8c7d8 
t = 36: 5933fc99 ab7c32ee 3a6b261f 4c71d5aa_ 16bcc90b 
t= 37: 43f87ae9 5933fc99 aadf0cbb 38a6b261f 4c71d5aa 
t = 38: 24957f22 43f87ae9 564cff26 aadfOcbb 3a6b261f 


t = 39: 
t = 40: 
t= 41: 
t = 42: 
t = 43: 
t = 44: 
t = 45: 
t = 46: 
t = 47: 
t = 48: 
t = 49: 
t = 50: 
t= 51: 
t = 52: 
t = 53: 
t = 54: 
t = 55: 
t = 56: 
t = 57: 
t = 58: 
t = 59: 
t = 60: 
t= 61: 
t = 62: 
t = 63: 
t = 64: 
t = 65: 
t = 66: 
t = 67: 
t = 68: 
t = 69: 
t = 70: 
t= 71: 
t= 72: 
t= 73: 
t = 7A: 
t = 75: 
t = 76: 
t= 77: 
t= 78: 
t= 79: 


adeb7478 
d70e5010 
79bcfb08 
f9bcb&de 
633e9561 
98cleab4 
c6ea241le 
a2ad4f02 
c8a69090 
88341600 
7e846f58 
86e358ba 
8d2e76c8 
ce892e10 
edea95b1 
36d1230a 
776c3910 
a681b723 
ac0a794f 
f03d3782 
9ef775c3 
36254b13 
4080d4dc 
2bfaf7a8 
513f9ca0 
e5895c81 
1037d2d5 
14a82da9 
6d17c9fd 
2c7b07bd 
fdf6efff 
112b96e3 
84065712 
ab89fb71 
c5210e35 
352d9f4b 
1la0e0e0a 
d0d47349 
ad38620d 
d3ad7c25 
8ce34517 


2495722 
adeb7478 
d70e5010 
79bcfb08 
f9bcb8de 
633e9561 
98cleab64 
c6ea241e 
a2ad4f02 
c8a69090 
88341600 
7e846f58 
86e358ba 
8d2e76c8 
ce892e10 
edea95b1 
36d1230a 
776c3910 
a681b723 
ac0a794f 
f03d3782 
9ef775c3 
36254b13 
4080d4dc 
2bfaf7a8 
513f9ca0 
e5895c81 
1037d2d5 
14a82da9 
6d17c9fd 
2c7b07bd 
fdf6efff 
112b96e3 
84065712 
ab89fb71 
c5210e35 
352d9f4b 
1la0e0e0a 
d0d47349 
ad38620d 
d3ad7c25 


50feleba 
89255fc8 
2b7addle 
35c¢39404 
le6f38ec2 
be6f2e37 
58cfa558 
26307a99 
blba8907 
a8ab53c0 
3229a424 
220d0580 
lfallbd6 
alb8d62e 
234b9db2 
33a24b84 
7b7aad6c 
8db448c2 
1ddb0e44 
e9a06dc8 
eb029e53 
bcOf4de0 
e7bddd70 
cd8952c4 
10203537 
Oafebdea 
144fe728 
79625720 
440df4b5 
452a0b6a 
5b45f27f 
Ableclef 
ff7dbbff 
c44ae5b8 
al0195c4 
6ae27edc 
7148438d 
cd4b67d2 
86838382 
74351cd2 
6b4e1883 
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564cff26 
50feleba 
89255fc8 
2b7addle 
35¢39404 
le6f38ec2 
be6f2e37 
58cfa558 
26307a99 
blba8907 
a8ab53c0 
3229a424 
220d0580 
lfallbd6 
alb8d62e 
234b9db2 
33a24b84 
7b7aad6c 
8db448c2 
1ddb0e44 
e9a06dc8 
eb029e53 
bc0f4de0 
e7bddd70 
cd8952c4 
10203537 
Oafebdea 
144fe728 
79625720 
440df4b5 
452a0b6a 
5b45f27£ 
Ableclef 
ff7dbbff 
c44ae5b8 
al0195c4 
6ae27edc 
7148438d 
cd4b67d2 
86838382 
74351cd2 


aadf0cbb 
564cf26 
50feleba 
89255fc8 
2b7addle 
35c¢39404 
le6f38ec2 
be6f2e37 
58cfa558 
26307a99 
blba8907 
a8ab53c0 
3229a424 
220d0580 
lfallbd6 
alb&d62e 
234b9db2 
33a24b84 
7b7aad6c 
8db448c2 
1ddb0e44 
e9a06dc8 
eb029e53 
bcOf4de0 
e7bddd70 
cd8952c4 
10203537 
Oafebdea 
144fe728 
79625720 
440df4b5 
452a0b6a 
5b45f27£ 
Ableclef 
ff7dbbff 
c44ae5b8 
al0195c4 
6ae27edc 
7148438d 
cd4b67d2 
86838382 
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We just finished the processing of M[1]. The exterior round state array 
becomes 


Ho = 67452301 + 8ce34517 = f4286818, 
A, = efcdab89 + d3ad7c25 = c37b27ae, 
Hy = 98badcfe + 6b4e1883 = 0408 f581, 
Hz = 10325476 + 74351cd2 = 84677148, 
Hy = c3d2e1 f0 + 86838382 = 4a566572. 


Let us attack now the second exterior round. The words of the block M[2] are 


0} = 00000000, 
1] = 00000000, 
2] = 00000000, 
3] = 00000000, 
4] = 00000000, 
5] = 00000000, 
6] = 00000000, 
7| = 00000000, 
8] = 00000000, 
9} = 00000000, 
10] = 00000000, 
1 
1 


| = 00000000, 
| = 00000000, 
13] = 00000000, 
14] = 00000000, 
15] = 000001c0, 


0 
1 
2 


SSSR SS SSR SSS SaaS 


The values for A, B, C, D, E after the t-th interior round are now the following: 


A B C D E 
t=0: 2df257e9 £4286818 bOdec9eb 0408f581 84677148 
t=1: 4d3dc58f 2df257e9 3d0ala06 bOdec9eb 0408f581 
t= 2: c352bb05 4d3dc58f 4b7c95fa 3d0ala06 bOdec9eb 
t= 3: eef743c6 c352bb05 d34f7163 4b7c95fa 3d0ala06 
t= 4: 41¢34277 eef743c6 7O0d4aecl d34f7163 4b7c95fa 


t= 5: 5443915¢ 41e34277 bbbddOfl 7Od4aecl d34f7163 
t=6: e7fa0377 5443915c d078d09d bbbddOfl 70d4aecl 
t= 7: c6946813 e7fa0377 1510e457 d078d09d bbbddOfl 


t= 8: fddeldel c6946813 f9fe80dd 1510e457 d078d09d 
t=9: b8538aca fddeldel fladSla04 f9fe80dd 15100e457 


t=10: 
t=11: 
G12: 
t=13: 
t=14: 
t=15: 
t=16: 
$=17: 
t=18: 
t=19: 
t= 20: 
t= 21: 
t= 22: 
t= 23: 
t= 24: 
t= 25: 
t= 26: 
t= 27: 
t= 28: 
t= 29: 
t= 30: 
t=31: 
t= 32: 
t= 33: 
t= 34: 
t= 35: 
t= 36: 
t= 37: 
t= 38: 
t= 39: 
t= 40: 
t=Al: 
t= 42: 
t=43: 
t=44: 
t=45: 
t= 46: 
t=A7: 
t=48: 
t= 49: 


6ba94f63 
43a2792f 
fecd7bbf 
a2604ca8& 
258b0baa 
d9772360 
5507db6e 
ad 1b58be 
c2eb709f 
d8992153 
37482f5f 
ee8700bd 
9ad594b9 
8fbaadb9 
88fb5867 
eec50521 
50bce434 
5c416daf 
2429be5f 
Oa2fb108 
17986223 
8a4af384 
6b629993 
£15f04£3 
295cc25b 
696da404 
cef5ael2 
87d5b80c 
84e2a5f2 
03bb6310 
c2d8f75f 
bfb25768 
28589152 
ecld3d61 
3caed7af 
c3d033ea 
7316056a 
46f93b68 
dc8e7f26 
850d411c 


b8538aca 
6ba94{63 
43a2792f 
fecd7bbf 
a2604ca8 
258b0baa 
d9772360 
5507db6e 
ad1b58be 
c2eb709f 
d8992153 
3748 2f5f 
ee8700bd 
9ad594b9 
8fbaadb9 
88fb5867 
eec50521 
50bce434 
5c416daf 
2429be5f 
0a2fb108 
17986223 
8a4af384 
6b629993 
£15f04£3 
295cc25b 
696da404 
cefd5ael2 
87d5b80c 
84e2a5f2 
03bb6310 
c2d8f75f 
bfb25768 
28589152 
ecld3d61 
3caed7af 
c3d033ea 
7316056a 
46f93b68 
dc8e7f26 


7£778778 
ael4e2b2 
daea53d8 
d0e89e4b 
ffb35eef 
2898132a 
8962c2ea 
365dc8d8 
9541f6db 
2946d62f 
fObadc27 
£6264854 
cdd20bd7 
7Tbalc02f 
66b5652e 
63eea96e 
e23ed619 
7bb14148 
142f390d 
d7105b6b 
c90a6f97 
028bec42 
c5e61888 
2292bcel 
dad8a664 
fc57c13c 
ca573096 
1a5b6901 
b3bd6b84 
21f56e03 
al38a97c 
O0eed8c4 
f0b63dd7 
2fec95da 
8al62454 
7b074f58 
cf2bb5eb 
b0f40cfa 
9ccd5815a 
11lbe4eda 
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fla51a04 
7£778778 
ael4e2b2 
daea53d8 
d0e89e4b 
ffb35eef 
2898132a 
8962c2ea 
365dc8d8 
9541f6db 
2946d62f 
fObadc27 
{6264854 
cdd20bd7 
7balc02f 
66b5652e 
63eea96e 
e23ed619 
7bb14148 
142f390d 
d7105b6b 
c90a6f97 
028bec42 
c5e61888 
2292bcel 
dad8a664 
fc57c13c 
ca573096 
1a5b6901 
b3bd6b84 
21£56e03 
al38a97c 
O0eed8c4 
f0b63dd7 
2fec95da 
8al62454 
7b074£58 
cf2bb5eb 
bO0f40cfa 
9cc5815a 


f9fe80dd 
fla51a04 
7£778778 
ael4e2b2 
daead53d8 
d0e89e4b 
ffb35eef 
2898132a 
8962c2ea 
365dc8d8 
9541f6db 
2946d62f 
fObadc27 
{6264854 
cdd20bd7 
7balc02f 
66b5652e 
63eea96e 
e23ed619 
7bb14148 
142f390d 
d7105b6b 
c90a6f97 
028bec42 
cd5e61888 
2292bcel 
dad8a664 
fc57c13c 
ca573096 
1a5b6901 
b3bd6b84 
21f56e03 
al38a97c 
O00eed8c4 
f0b63dd7 
2fec95da 
8al62454 
7b074f58 
cf2bb5eb 
bOf40cfa 
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t=50: 7e4672c0O 850d4llc b7239fc9 1lbe4eda 9cc5815a 
t=51: 89fbd41d 7e4672c0 21435047 b7239fc9 11lbe4eda 
t=52: 1797e228 89fbd4ld 1f919cb0 21435047 b7239fc9 
t=53: 431d65be 1797e228 627ef507 1f919cb0 21435047 
t=54: 2bdbb8cb 431d65bc O5e5f88a 627ef507 1f919cb0 
t=55: 6da72e7f 2Zbdbb&cb 10c7596f O5e5f88a 627ef507 
t=56: a8495a9b 6da72e7f caf6ee32 10c7596f O05e5f88a 
t=57: e785655a a8495a9b db69cb9f caf6ee32 10c7596f 
t=58: 5b086c42 e785655a eal256a6 db69cb9f caf6ee32 
t=59: a65818f7 5b086c42 b9e15956 eal256a6 db69cb9f 
t=60: T7aabl01b a65818f7 96c21b10 b9e15956 eal256a6 
t=61: 93614c9c Taab101b e996063d 96c21b10 b9e15956 
t=62: f66d9bf4 93614c9c deaac406 e996063d 96c21b10 
t=63: d504902b f66d9bf4 24d85327 deaac406 e996063d 
t=64: 60a9da62 d504902b 3d9b66fd 24d85327 deaac406 
t=65: 8b687819 60a9da62 f541240a 3d9b66fd 24d85327 
t=66: 083e90c3 8b687819 982a7698 [541240a 3d9b66fd 
t=67: f6226bbf 083e90c3 62dale06 982a7698 f541240a 
t=68: 76c0563b f6226bbf c20fa430 62dale06 982a7698 
t=69: 989dd165 76c0563b fd889aef c20fa430 62dale06 
t=70: 8b2c7573 989dd165 ddb0158e fd889aef c20fa430 
t=71: aelb8e7b 8b2c7573 66277459 ddb0158e fd889aef 
t=72: cal840de aelb8e7b e2cbld5c 66277459 ddb0158e 
t=73: 16f8babb cal840de eb86e39e e2cbld5c 66277459 
t=74: d28d83ad 16f3babb b2861037 eb86e39e e2cbld5c 
t=75: 6bc02dfe d28d83ad cSbceeae b2861037 eb&6e39e 
t= 76: d3a6e275 6bc02dfe 74a360eb cSbceeae b2861037 
t=77: da955482 d3a6e275 YafO0Ob7f 74a360eb cdSbceeae 
t=78: 58cOaac0 da955482 74e9b89d YafOOb7f 74a360eb 
t=79: 906fd62c 58cOaac0 b6a55520 74e9b89d YafO00b7f 


This ends the treatment of M[2]. The (final) state of the buffer H after the 
second round will be 


Hy = £4286818 + 906fd62c = 84983644, 
Ay = c37b27ae + 58c0aacd = 1c3bd26e, 
Az = 0408f581 + b6a55520 = baaedaal, 
3 = 84677148 + 74e9b89d = £95129e5, 
4 = 4a566572 + 9af00b7f = e54670f1. 


We sum up: 


If M = “abcdbcdecdefdefgefghfghighijhijkijkljklimklmnlmnomnopnopq” in 
ASCII, then SHA-1(/)=84983e44 1c3bd26e baaedaal f95129e5 e54670f1 
(in hex). 
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Exercises 
(1) Recall: The Boolean operations on words are executed bitwise — i.e. 32 


times in parallel. 
(a) Compute f(B,C,D) = (BAC) V ((=B) A D) for B = 69, C = f2, 


D = 57. 
(b) Compute g(B,C, D) = (BAC) V(BAD)V(CAD) for B=a3, C = 02, 
D=fe. 


(2) Write S°, the 5-position cyclic left shift on 32-bit words, by means of 
arithmetical operations (on the integer values of the words) and of Boolean 
operations (position by position). 

(3) Consider the following hash algorithm u-sha which is an academic pocket 
version of SHA-1: 


The words will have four bits (instead of 32 bits); the blocks will have 16 bits 
(instead of 512 bits). 

We shall treat messages M of length | < 2°. 

M* = M[1|M[2]---M[n] is obtained as in SHA-1; here, the last block M[n] 
is the length of M (in binary notation, over 16 bits). 

The message digest m = u-sha(M/) will have 20 bits (5 words). 

We shall have exterior rounds for the successive treatment of the M/i], and 
interior rounds which carry out this treatment. Every exterior round will be 
composed of four interior rounds (t = 0,1, 2,3). 

The algorithm imitates SHA-1, with the following modifications: 


(i) Initialization: Hp = 9, H, = 5, Hp =a, H3 =c, Hy =3. 
(ii) M{i] _ WW,W2W3 (16 bits = 4 words) 
There are no other words to compute. 
(iii) The logical functions: 


(BAC) Vv ((-B) A D), t=0, 
BeceD, t=1, 
to (BAC)V(BAD)V(CAD), t=2, 
BoeCeD, t= 3, 


(iv) The interior round constants: 
Reo F426 FGGAG,. RS. 


(v) The interior round transformation is defined as in SHA-1; note only that 
So 8 and thats? =S* 
(a) Let 1 =bebad (in hex). Compute m = u-sha(M). 
(b) Let M= faced (in hex). Compute m = pt-sha(M). 


(4) Back to the algorithm SHA-1. Consider M = “abc” in ASCII: 
M = 011000010110001001100011 = 616263. 
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(a) Write down M* = M[1] = WoW1--- Wis. 
(b) Compute the content of the buffer ABCDE after two interior rounds 
(t= 0,1). 
(5) In the situation of exercise (4), introduce an error in the leading position: 
M = 616263 +> M’=e16263. 


What are the binary positions of the buffer ABCDE which are affected by 
this error after four interior rounds (i.e. after t = 3)? 


2.4.2 DSA: Digital Signature Algorithm 
Generalities 


DSA is the principal standard for digital signatures. It is a kind of public key 
cryptosystem on message digests. Every user will dispose of a private (secret) 
multi-key and a public key. The private keys have a constant component 
(identification of the user) and a variable component (identification of the act 
of signature), which changes with every new signature. The public keys are 
constant (during a certain period) and may even be shared by some group of 
users. The private keys act (as parameters) on the signature algorithm; the 
public keys act on the verification algorithm (of the signatures). The signature 
generation (for a message M) consists in computing a “ciphertext” which is 
a couple of 160-bit words. 

The first component of the signature is “the (secret) identifier of the act 
of signature, in an envelope”. The second component of the signature is “the 
signature attached to the document”; more precisely, it is a ciphertext com- 
puted from the message digest of the document M, which will be masked by 
means of the action of the private key of the signatory (i.e. of the signatory’s 
identifier as well as the identifier of the act of signature). 

The verification process is an implicit decryption: One verifies if the first 
component of the signature is correctly an (implicit) function of the second 
component. 


The scheme of functioning for DSA 
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Signature Generation 


Signature Verification 


Private 
=> 
Key 


Message 


V 
SHA-1 


V 
Message Digest 
V 


DSA Sign 
Operation 


Digital 
=> 
Signature 


Digital 
=> 
Signature 


Received Message 


SHA-1 


Message Digest 


DSA Signature 
Verification 


Yes — Signature Verified 
No — Verification Failed 


Public 
< 
Key 


The Functioning of DSA 


The Individual Parameters. 


The private keys and the public keys of a user A stem from the following 


parameters: 


(i) p: a prime number such that 21073 < p < 21074 
(ii) g: a prime number dividing p — 1, with 21°9 < q < 21 
(iii) g: an integer 1 < g < p such that g? = 1 mod p 


e P, q, g are public and are often common to a group of users 


(iv) ©=a24:1< 2 <4, the private key of the user A 
Y = ya = g” mod p: the public key of the user A 
(v) k= ka: 1 <k <q, the private key of the user A, designed to mark the 
act of signature 


e x =x, and y = ya are in general kept fixed for a period of time. k = k4 


must be regenerated for each signature 


The algorithms of (pseudorandom) generation of the values p, qg, « and k will 


be presented in a moment. 


Remark 


(1) How is it possible to publish y = g* mod p whilst keeping secret the 
value x? We note that we are confronted with the problem of computing 
the discrete logarithm (to the base g) in Z/pZ. The situation is similar 
to the problem of factoring big integers in RSA: the algorithms which 
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compute discrete logarithms have the same slowness as those which factor. 
This explains the size of p (familiar to us by the discussions on RSA) which 
has to guarantee the algorithmic infeasibility of finding (in a reasonable 
time) « from y = g” mod p. 

Let’s insist: The security of DSA (as a public key system) depends on the 
choice of the size of p. The appearance of g — which is of sensibly smaller 
size — comes from the needs of economy (in space and in time).® We shall 
see in the sequel that a lot of computations (and of storage) will be thus 
restricted to 160-bit format (instead of 1024-bit format). 


The Signature Generation 


The user A would like to sign a message (a document) M (of length | < 2° 
bits). 


A first generates the private key of the act of signature k = ky. 

A computes m = SHA — 1(M), the message digest of M. 

Then it computes the signature’ o = (r,s), where r = (g* mod p) mod gq, 
s=(k-}(m+ar)) mod q® 

If r = 0 or if s =0, then A has to generate a new value for k = kg. 


The Signature Verification 


For a thorough understanding of the authentication procedure, we have to 
look first a little bit closer at the mathematical structure of the signature 
ao = (r,s). Actually, there is a implicit function relation 


r= F(r,s,m) 


which does not depend on the knowledge of the private parameters (x, k) of 
the signatory A. 


6 


7 
8 


Encouraging observation: Compute simply in R: 


k m+ x2r 


k % 


y=Q". 


To the intuitive reader: The complexity of the situation depends altogether on p. 


Compare with the geometry of a plane defined by 26 linear equations in R”*. 
There is no permanent place in the world for ugly mathematics. — G.H. Hardy. 
We identify the binary word m = mimz2---mieo with the integer m,2)°9 + 
mo2'8 +--+. + migo; k71 will mean the integer u: 1 < u < q, such that 
ku = 1 mod gq. 
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Put ‘ 
m __ m 
WL ag, m+ar? 
Us = a kr 
2~ 3 ~ m+tar* 
—km_ kor ; th iy 
Then: gy" = gmter -gmter = gk = r, This observation indicates the 


right way for the signature validation procedure. In fact: 


Proposition Let o = (r,s) be the signature of the message M by the user A, 
and let m= SHA-—1(M). Let us put 

w = s—! mod q 

u, = mw mod q 

ug = rw mod q 

v = (gy mod p) mod q 
Then, v=r. 


Proof We observe first that g* = g@ modp <= > e, =e€2modq. 

This is an immediate consequence of the fact that the order of g modulo p is 
precisely equal to q. The remainder is easy: gy"? mod p = g("**")” mod p. 
But: (m+ ar)w = ksw =k mod qi. g("+*")” = g* mod p . This yields the 
claim. 


We get the procedure of signature verification: 


Let M be the message (the document) signed by the user A, m = 
SHA-1(M), and let o = (r,s) be the signature. Let M’ be the received message 
(the received document), and let o’ = (r’,s’) be the transmitted signature. 
Finally, let (y,p,q) be the public key of A. 

One computes 


—~ m' =SHA-1(M’), 

— w=(s')-! mod gq, 

— u,=m'w mod gq, 

— u2=r'wmodgq, 

— v=(g“'y” mod p) mod gq. 


Ifv = r’, then the signature is verified. If v 4 r’, then the message is considered 
to be invalid. 


Exercises 


(1) How can g be found (an element of order g in (Z/pZ)*)? 


Suppose that p and q are already at hand, and let e = ies. 


Show that the following algorithm will find an appropriate value for g. 


a. Let h: 1 <h< p-—1 be the next test value. 
b. Compute g = h® mod p 
c. If g = 1, return to a. ; otherwise, g will do. 
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(2) A class room version of DSA. 
We will choose gq = 101, p = 78q + 1 = 7879. 
(a) Show that 3 is a generator of (Z/7879Z)*. 
(b) We may take g = 37° mod 7879 = 170. Let us choose x = 75. Suppose 
that the message digest of M is m = 1234. Our signature key will be 
k, = 50. Compute the signature o = (r,s). 
(c) Verify the signature. 
(3) The situation of the preceding exercise. Now, consider m = 5,001 and 
k = 49. Compute, once more, the signature o = (r,s), and carry out the 
validation. 


2.4.3 Auxiliary Algorithms for DSA 


The algorithm DSA is a great consumer of big numbers — a lot of them prime 
numbers (like the public p and q), but a great portion not necessarily so (like 
the private 2 and k) — which have all to be supplied by pseudorandom gen- 
erators, i.e. in simulating methodically a kind of “chance under control”. We 
shall outline briefly the indispensable details of these algorithmic providers of 
big numbers, while respecting the logic of attribution: the public keys are — in 
this sense — primary, the private keys secondary. 


The Miller—Rabin Primality Test 


With the birth of the arithmetical cryptosystems, where the manipulation of 
big prime numbers became commonplace, the efforts of the mathematicians 
to provide a formidable palette of primality tests were plainly successful: The 
inheritance of the last centuries was too rich. The test of Miller—Rabin, which 
we shall present in a moment, is the little grey duck in a family of white swans: 
On the one hand, it is very easy to understand (modulo a statistical infor- 
mation which can be established via several exercises in elementary number 
theory), on the other hand, since it refers to a probabilistic argument, it will 
not be joyously accepted by the mathematical pencil-and-paper community. 
But ultimately, for a pragmatic and unscrupulous practitioner, it is one of the 
handiest available primality tests. 


(1) The test SPP(n,b) = me 


[SPP(n, b) = n is a strong pseudoprime for the base b. | 

Let us fix n = 2°m +1, with a > 1, m odd. n will be the candidate — a 
(big) odd number that we would like to be prime. The integer b is chosen 
randomly between 1 and n:1<b<n. 


Then: SPP(n,b)=yes <= > either b&” =1 mod n or b™ = —-1 mod n 
or D2" = —1 mod n or b4” = —1 mod n...or 6?” -™ = —1 mod n 
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(2) The justification of the test SPP: 


wm 


WN 


If p is a prime number, and if b is an arbitrary integer between 1 and p, 
then we necessarily get: SPP(p,b) = yes. 

The argument: 

In the given situation, the (little) theorem of Fermat yields: 

b?-! = 1 mod p. Write p — 1 = 2m with a > 1 and m odd. 


[Lemma: The polynomial X?~! — 1 admits the following factorization: 


XP = (X™—1)(X™ 41)(X2" 41)(X4M 4.1). (X27 -™ 4:1). 


Proof Recursion on a > 1. 


Thus, P-1 — 1 = (6 — 1)(b™ + 1)(b?™ + 1)(04" +.1)--- (p2"*m +1). 

p divides b?—! — 1, hence p divides at least one of the a+ 1 factors above: 
b™ = 1 mod nor b” = —1 mod nor b?” = —1 mod nor O*" = —1 mod n 
...or b2"'™ = —1 mod n; ie. SPP(p,b) = yes for every b, 1 <b<p. 
The fundamental result for the Miller—Rabin test: 

If n is composite, then SPP(n,b) = yes for not more than 25% of the b 
between 0 and n. In other words: 


The answer of the test is against its intent with a probability < 3. 


Let then n be our candidate (which we want to be prime). We shall ran- 
domly choose b between 1 and n. If SPP(n,b) = yes, then the probability 
of n not being prime is < i 

The Miller—Rabin test (1980). 

Let n (a positive odd integer) be the candidate of our primality test. 

We choose } randomly between 1 and n. If SPP(n,b) = yes, then n has 
passed the first round of the test: n is not prime with a probability < i. 
Let us choose, a second time, ) randomly between 1 and n. If SPP(n, b) = 
yes, then n has passed the second round of the test: n is not prime with 
a probability < % and so on. If n passes N rounds of the test SPP, then 
the probability of n not being prime is < (;)%. One fixes N > 50. If n 
passes N times the test SPP, one considers n to be a prime number. 


Exercises 


(1) For N > 2, let Gy = {a € (Z/NZ)* : aN-1 = 1}. 


(a) Show that Gy is a subgroup of (Z/NZ)*. 
(b) Determine the order (the cardinality) of Gy in case N = 1763. 
(c) Find explicit representatives for G1763. 


(2) N = 20051533153 = 100129 - 200257. 


Find two integers b, 1 < b < N, such that SPP(N,b) = yes and also two 


integers b, 1 < b < N, such that SPP(N,b) = no. 
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Generation of the Prime Numbers p and q for DSA 
Let us begin with some remarks on the density of the prime numbers (in N). 


Remark (1) There are 78,497 odd prime numbers up to 10°. This gives a 
density of a (Note that — = a — ca 

In general, we dispose of the following famous result (Hadamard, de la 
Vallée-Poussin, 1896): 

Let z(a) be the number of primes < «, and let 


(2 


New 


Then: 


Consider a (big) integer z € N ; then the density of the prime numbers 
of the size of z is approximately equal to ~ Example: For z = 101, 
the corresponding density will be aa This means that we must — in 
average — look at 115 consecutive odd candidates in order to find a prime 


number of 100 decimal positions. 


Back to DSA. 
We have to generate — as public parameters of a user or of a group of 
users — two prime numbers p and q such that 


(1) 9159 <q< 2490, 
(2) 91023 <p< 91024 
(3) q divides p— 1. 


— 
wo 
wN 


— (1)The generation of g is done by SHA-1 (perverted into an auxiliary 
algorithm), using the numerical content of a buffer X’ (for SEED), defined 
by the user. 

— (2) and (3) The same value of X’ (as that used in (1)) will serve to compute 
some “prototype” X of p: 21073 < X < 21024 , 


Then, we have to “trim” X in order to obtain the prime number p = 1 mod 2q. 


Recall As always until now, we shall make no notational distinction between 
a binary word 2 22---%, of length o and the integer 2 = 7,27~! + 2927-7 + 
-+++24 (0 <a < 27) which admits the binary word as its notation. 

We insist: The lecture is like the usual decimal lecture; the most significant 
bit is to the left, the least significant bit is to the right. 


(1) The small prime number g: 
(la) The user initializes ©’ (o = the length (of the content) of ©’; 0 > 160). 
(1b) U = SHA-1(’) 6 SHA-1((2'+ 1) mod 27) 
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(Ic) q=2°VUV1_ (the first and the last bit are now equal to 1) 
9159 < g < 2160 

(1d) Finally, g has to pass a robust primality test (the probability of a 
non-prime number passing the test must be < 50): If g is not prime, 
return to (la). 

(2) The big prime number p: 
We begin with a crude approach which neglects the divisibility constraint 
(q has to divide p — 1): 

(2a) Let C be the counter for the iterated runs to find p using the spe- 
cific seed 7 which helped to generate q. We initialize C = 0. At the 
same time, we initialize a shift constant (for the computation to come) 
D=2. 


(2b) Now, we compute seven digits of a 21©° 


-adic development: 


Vie = SHA-1[(8+D+k)mod29] 0<k<6. 


This gives W = Vg - 296° + Vz - 2809 4 VY, - 2640 4 Ya . 2480 + Vy - 2320 4 
V, - 2169 4. Vo. Then X = 21073 + W (so we get 21093 < X < 21074), 
Finally, we must try to get p “with the help of X”: 

(2c) Let then c= X mod 2q and let p= X—(c—1) (hence p = 1 mod 2g). 
If p < 21973, we go to (2d). 
Otherwise, we perform a robust primality test on p. If p passes the 
test, we save the values of ’ and of C (for use in certifying the proper 
generation of p and of q). Else, we go to (2d): 

(2d) Let us increment: C+ C+1,D-D+7. 


If C > 2'? = 4096 (we have tested in vain too many candidates on the base 
of the seed 3, etc.), then we start again — beginning with (la). 
Else, we go back to (2b) — q remains valid, still a little effort for p ... 


Random Number Generators for the DSA 
The Ausiliary Function G(t, c) 


The function G(t,c) will accept arguments t of 160 bits and arguments c of 
160 bits (version DES) or of 160-512 bits (version SHA-1) in order to produce 
values that are 160-bit words. 

The SHA-1 version. There will be only one exterior round of the algo- 
rithm SHA-1: t = totitotst, (5 times 32 bits) initializes the buffer H = 
Ay H,H2H3H, of SHA-1. The second argument c (which will have b bits: 
160 < b < 512) defines the 16 first words (the “round keys” for the 16 first 
interior rounds) in the following way: WoW, ---Wis =c 0---0. 

512—b times 

Note, by the way, that the block of 512 bits above is not of the form 

M* = M[1]. G(t,c) will then be the final state of H = H)H, H2H3H4. 
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The DES version. Let’s begin with a (notational) 


Definition Let a1, a2, b, bg be four 32-bit words. 
DESy, ».(@1, 42) = DES‘ (A), where 


A= aja (plaintext of 64 bits) 


K =0{bo (key of 56 bits) 
bi = the 24 last bits of by 


Let now t and c be given (of 160 bits each); compute G(t, c) (of 160 bits): 


(i) t = tytetstats, c = c1c2c3C4c5 ( segmentation into five words, of 32 bits 
each) 
(iii) For 1 <7 <5 we put: 
a1 = %, 
Q2 = Xj mod 5)+1 DY L((i43) mod 5)+1) 
by = C((443) mod 5)-+15 
bz = C((442) mod 5)-+15 
Yi,1Yi,2 = DES, 5 (a1, az). 
(iv) G(t,c) = 2122232425 with 2; = yi,1 8 Y((i41) mod 5)4+1,2 © Y((i+2) mod 5)+1,1 
1<7i<65 (for example z3 = y3,1 © Y5,2 © Yi). 


The algorithm for computing m values of the private key x. b will denote the 
size of c — the second argument of the auxiliary function Gt, c) 
(160 < 6 < 512). We shall proceed this way: 


1. Choose a new, secret value for xj) (= the seed-key — of length 6). 
2. t = 67452301 efcdab89 98badcfe 10325476 c3d2elf0 (the “universal” 
initial value for SHA-1). 
3. Computation of m copies for @: 20, %1, .--, Lm-1: For 7 = 
0,1,2,...,m-—1 do: 
(i) Define ox; (=a-seed) by an optional user input (of length b). 
(ii) For « = 0,1 do (computing twin-values, for every 7): 
a. c; = (%%) + 72;) mod 2°, 
b. Wi = Gt, Ci), 
c. Xy) = (1+2) +w;) mod 2° (the seed-key will change 2m times). 
(iii) x; = (wo||w1) mod gq (concatenation of wo and w1). 


The algorithm for precomputing m values of the key of signature k for several 
documents. Let us sum up the situation: p, g and g, the public parameters of 
the user, have already been generated. b will have the same meaning as before. 
One wants to precompute k, k~! and r for m messages at a time. The steps 
of the algorithm are the following: 


1. Choose a secret initial value kj) (once more the seed-key — of length 6). 
2. t=efcdab89 98badcfe 10325476 c3d2e1f0 67452301 (a cyclic word-shift of 
the standard initial value for SHA-1) 
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3. For 7 = 0,1,2,...,m—1 do: 
a. Wo = Git, ky), kw) = (1 kw) wo) mod 2°. 
w, = Gt, ky), kw) = (1 kw) w 1) mod vhs 
b. k = (wo||w1) mod gq. 
c. Compute he =k7' mod q. 


d. Compute r; = (g* mod p) mod gq. 
4. Let now Mo, My, Mo, ..., Mm_1 be the next m messages to sign. We 
continue this way: 
e. h= SHA-—1(M;) 
f. sj = (ky *(h +ar,;) mod q 
g. (rj, 8;) = the signature for M;. 
5. t=Ah. 
6. Go to 3. 


We note: Step 3 permits pre-computation of the quantities needed to sign the 
next m messages. Step 4 can begin whenever the first of these m messages 
is ready (and the execution of step 4 can be suspended whenever the next of 
the m messages is not ready). As soon as steps 4 and 5 have been completed, 
step 3 can be executed, and the result saved until the first member of the next 
group of m messages is ready. 


An Example of DSA 


We shall consider a “soft version”, with 2°'! < p< 2°)”, 

First some remarks on the initializations chosen for the auxiliary algo- 
rithms: 

The seed-value »’ of the algorithm which computes p and q was chosen to 
be 3’ = d5014e4b 60ef2ba8 b6211b40 62ba3224 e0427dd3. 

With this value for 2’, the algorithm did produce p and gq when the counter 
C was at 150 (recall that we save the values of ©’ and of C in order to certify 
the correct generation of p and q). 

The auxiliary function G(t,c) was constructed in version SHA-1, with b = 
160. 
All the seed-keys had the same length of 160 bits. 

Xp) = bd029bbe 7f51960b cf9edb2b 61f06f0F ebdas8b6, 

t = 67452301 efcdab89 98badcfe 10325476 c3d2e1f0, 

x=G(t, L(p)) mod q, 

ky) = 687a66d9 0648f993 867e121f 4ddf9ddb 01205584, 

t=efcdab89 98badcfe 10325476 c3d2e1f0 67452301, 

k = G(t, ki)) mod q®. Now we are ready to present our example: 

h = 2 (his the value introduced in exercise (1) on DSA), 


° The watchful reader will realize that the auxiliary algorithms appear here in 
an older version — without splitting into wow1-twins and some eager reductions 
modulo q. 
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p = 8df2a494 492276aa 3d25759b b06869cb eac0d83a fb8d0cf7 cbb8324f 
0d7882e5 d0762fc5 b7210eaf c2e9adac 32ab7aac 49693dfb £83724c2 ec0736ee 
31c80291, 

gq = c773218c 737ec8ee 993b4f2d ed30f48e dace915f. 

5} = 626d0278 39ea0al3 413163a5 5b4cb500 299d5522 956cefcb 3bff10f3 
99ce2c2e 71cb9de5 fa24babf 58e5b795 21925c9c c42ce9f6f 464b088c c572af53 
e6d78802. 

x = 2070b322 3dba372f delcOffc 7b2e3b49 8b260614. 

k = 358dad57 1462710f 50e254cf 1a376b2b deaadfbf. 

k—! = 0d516729 8202e49b 4116acl10 4fc3f415 ae52f917. 

The message M was the ASCII equivalent of “abc” (cf exercise (4) on 
message digests). 

m= SHA —1(M) = a9993e36 4706816a ba3e2571 7850c26c 9cd0d89d, 

y = 19131871 d75b1612 a819f29d 78d1b0d7 346f7aa7 7bb62a85 Ibfd6c56 
75da9d21 2d3a36ef 1672ef66 Ob8c7c25 5cc0ec74 858fba33 £44c0669 9630a76b 
030ee333, 

r = 8baclab6 6410435c b7181f95 b1l6ab97c 92b341c0, 

s = 41e2345f 1f56df24 58f426d1 55b4ba2d b6dcd8c8, 

w = 9df4ece5 826be95f ed406d41 b43edc0b 1c18841b, 

uy = bf655bd0 46f0b35e c791b004 804afcbb 8ef7d69d, 

uz = 821a9263 12e97ade abcc8d08 2b527897 8a2df4b0, 

g"' mod p = 5l1b1bf86 7888e5f3 af6fb476 Idd016be fe667a65 aafc2753 
9063bd3d 2b138b4c e02cc0c0 2ec62bb6 7306c63e 4db95bbf 6f96662a 1987a21b 
e4ec1071 010b6069, 

y“2 mod p = 8b510071 2957e950 50d6b8fd 376a668e 4b0d633c 1e46e665 
5c611a72 e2b28483 bed52c74d 4b30de61 a668966e dc307a67 c19441f4 22bf3c34 
O8aebalf Oaddbec7, 

v = 8baclab6 6410435c b7181f95 b16ab97c 92b341c0. 


2.4.4 The Signature Algorithm rDSA 


The use of the cryptosystem RSA for digital signatures was standardized in 
September 1998 (by the ANSI=American National Standards Institute). 

We shall present a “minimal” version, skipping details on the auxiliary 
algorithms and on options of refinement. 


The Key Generation 


We have to 


— Choose the public exponent e (for the verification of the signature) 

— Generate two big prime numbers p and qg (which must be kept secret), 
then compute 
n = pg (which will be public) 
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— Compute the private exponent d, which will serve for the signature gener- 
ation. 


First, we shall fix the length of the binary notation of n equal to 1,024 (this 
is the smallest possible value accepted by the standard). Then, we put e = 3 
(or e = 216 + 1 = 65537), uniformly for all users (we are also permitted to 
produce e individually and randomly ...). 

The pseudorandom generation of p and of g (of 512 bits each) is controlled 
by a list of constraints similar to those which control the auxiliary algorithms 
of DSA. Note that the exponent e must be prime to p— 1 and to q—1. 

For protection of n against certain classical factorization algorithms, we 
have to respect two rules: 


(i) p+1 and g+1 must each have a big prime factor of size between 100 and 
120 bits. 

(ii) p and q must differ at least in one of the first 100 binary positions (i.e. 
[a Rae), 


For the private exponent d we demand that d > 2°! (if one knows that the 
exponent d is small, an attack against RSA is possible, etc.) 


The Signature Generation 


Let M be the message to be signed. 
First, we compute m = SHA-1(M), the message digest of the message. 
Then, we proceed to carry out an encapsulation of the message digest m: 


= 
Il 


header | padding)|digest| trailer 


m* will have 1024 bits (= the size of n). 
Header: (4 bits) 6 = 0110 


imes b = 1011 
Padding: (844 bits) pipe ae eS ae 


Digest: (160 bits) m = SHA—1(M) 


Trailer: (16 bits) 33cc = 0011001111001100 

(“cc” will always be the end of m*; “33” identifies SHA-1, the hash algo- 
rithm producing the message digest) 

Note that numerically we have 2102? < m* < 21073 and m* = 12 mod 16. 

We can now produce the signature: 


o =min{m** mod n,n — (m*4 mod n)} 
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Remark Numerically, we always get 2102? < ¢ < 21093, (Why? — an exercise !) 


The Signature Validation 


Let M’ be the transmitted message, and let o’ be the transmitted signature. 


Decryption: 
Compute first 7’ = (a’)® mod n. 
Then m’* = 7 if +r’ =12 mod 16 


n—T' if n—7’ =12 mod 16 


If neither case is true, one rejects a’. 


a’ is rejected, if m™ is not of the form 6 bbb...ba m’ 33cc. 


In case we are formally satisfied, we recover m’. 
Validation of the signature: 


Compute SHA-1(M’) and verify if m’ = SHA-1(M"). 


If the answer is affirmative, the signature is verified. Otherwise, we reject it. 


An Example 


We consider the case where e = 3. Our two prime numbers p and q, of 512 


binary positions each, are: 


p  d8cd81f0 35ec57ef e8229551 49d3bff7 0c53520d 769d6d76 646c7a79 
2el6ebd8 9fe6fcdSb 606b56f6 3eb11317 a8dccdf2 03650ef2 8d0cb9a6 


d2b2619c 52480f51 


q  ccl09249 5d867e64 065dee3e 7955f2eb c7d47a2d 7c995338 8f97dddc 
3elcal9c 35ca659e dc3d6c08 £64068ea fedbd911 27f9cb7e 


dc174871 1b624e30 b857caad 


dis computed in solving the congruence 3X = 1 mod ppcm(p—1, q-1 
lccda20b cffb8d51 7ee96668 66621b11 822c7950 d55f4bb5 bee37989 
a7d17312 e326718b e0d79546 eaae87a5 6623b919 b1715ffb d7f16028 
fc400774 1961c88c 5d7b4daa ac8d36a9 8c9efbb2 6c8a4a0e 6bce15b35 
8e528ala c9d0f042 beb93bca 16b541b3 3f80c933 a3b76928 
5c462ed5 677bfe89 df07bed5 c127fd13 241d3c4b 

n  acdicc46 dfe54fe8 9786672 664ca269 Od0ad7e5 003bc642 7954d939 
eee8b271 52e6a947 450d7fa9 80172de0 64d6569a 28a83fe7 Ofa840f5 
e9802cb8 984ab34b d5cle639 Yec21e4d 3a3a69be 4e676f39 5aafef7c 
4925fd4f aee9f9e5 e48af431 5df0ec2d b9ad7a35 Ob3df2f4 
d15dc003 9846dlac a3527bla 75049e3f e34f43bd 

Let us pass now to the signature of the document M = “abc” = 616263 


(in ASCII). 


10 By the Chinese Remainder Theorem, one immediately gets, for every a € Z which 
is prime to n = pq: aPP°™P-149-1) = 1 mod pq — and that’s what we need, etc. 
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m = SHA-1(M) —a9993e36 4706816a ba3e2571 7850c26c 9cd0d89d 


m 


* 


6bbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb 

bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb 
bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb 
bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb 
bbbbbbbb bbbbbbbb bbbbbbbb bbbaa999 

3€364706 816aba3e 25717850 c26c9cd0 d89d33cc 


m*4 mod n  aG6b496f4 a802af90 92f1f561 931d84db d0b943ef 34c102b9 


This value exceeds 


oO 


4dd5lab0 1e1054be 0e0572a1 fb2db034 5698838 82b74e44 
9f6c80c4 060fbcOf fbd3a9ca 9d66685b 90873007 d207c1d6 
4c692d01 11157bb9 76a4551le 72ddc83c 767a9d75 a4746c51 
9b73ce52 c2bfbdle 3c431d25 4fe8bb43 O8fead86 787f239f 
d2944390 da49de45 

5; the signature will be the opposite value mod n. 
61d35523 7e2a0586 6867110d 32f1d8d3 c5193f5c b7ac3892 b7fbe89d 
0d85db54 4e136a54 Idfcf752 Y7ea9ece 21f08558 I3bbf230 9I884e5e 
dac82edf ae44af04 53ab631c cba5c76e dd1l3cbd3 d51f37fe 40b9a5dd 
64835133 86f5c704 01687dfc 27d1ddaf 6edbd18c efad5cf8 17504c08 
£482d262 ad3577aa 2705aaf0 9056578 


Finally, we can verify the validity of the signature: 


7! 


We compute now 7’ = a? mod n. 

4116108b 2429942d 3dbcaab6 aa90e6ad 514f1c29 44800a86 
bd991d7e 332cf6b5 972Zaed8b 8951c8ed c45b7224 a9la9ade 
6cec842b 53ec853a 2dc470fc dc8ef790 1a062a7d e3066291 7e7eac02 
92abb37d 9ef433c0 8d6a4193 f32e3e2a 28cf3875 a2353071 fdflbe79 
4f83495b 932778fd 16dc176e 7de102c9 b298016f Oab20ff1 


It is nm — 7’, which is congruent to 12 mod 16; we thus obtain: 


m 


Ie 


6bbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb 
bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb 
bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb 
bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb bbbbbbbb 
bbbbbbbb bbbbbbbb bbbaa999 3e364706 816aba3e 25717850 
c26c9cd0 d89d33cc 


which contains m = SHA-1(M) at the right place. We can now accomplish 
the validation. 


2.4.5 ECDSA — Elliptic Curve Digital Signatures 


The theory of elliptic curves is perhaps the most distinguished domain of 
modern Pure Mathematics. Its nobility comes from a fascinating interaction 
between Analysis, Geometry, Algebra and Number Theory, which has created 
a formidable tapestry of deep and beautiful results. 

There also remains some precious crumbs for cryptography. 
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More precisely: Classically, an elliptic curve (over C) is defined by an 
equation of the form 
y =z? +ar+b 


where the cubic polynomial in x has only simple roots. 

The important property of elliptic curves for cryptography is the following: 
There exists a natural method — in geometric language — to put on such a curve 
the structure of an abelian group E. 

As a consequence, if you now consider the same situation over a finite 
field k, the corresponding (finite) group E will be a kind of complicated ad- 
ditive substitute for the multiplicative group k*, but with a less decipherable 
structure for the lorgnettes of cryptanalysis. 

If we then replace exponentiation by iterated addition, it will be easy to 
formulate a signature protocol in complete analogy with that of DSA. 

Our exposition will focus on the case of characteristic 2 (hence: 1+1 = 0). 
Why? 

The arithmetic of the fields Fam = Fe[x]/(p(~)) — where p(x) is an irre- 
ducible binary polynomial of degree m — harmonizes perfectly with the binary 
computer technology and simplifies considerably in case of irreducible trino- 
mials (pentanomials) p(x). 

But, until now, we have never seen groups of type (Fgm)* in public key 
cryptography. The reason is simple: In order to get the discrete logarithm 
problem to be difficult in the multiplicative group (Fom)*, one is obliged to 
raise considerably the degree m. On the other hand, the groups € = €(Fom) of 
elliptic curves over the fields Fam become pretty complicated — in the sense we 
need it — for already reasonable degrees m. Thus, the marriage will be logical: 
If we want to work, for digital signatures, in binary polynomial arithmetic, 
then we need the elliptic curves over the various fields Fam. 

The prize to pay is a certain alienation of the “elliptic formulary”: the case 
of characteristic 2 is not natural with respect to classical theory. 


The algorithm ECDSA will work precisely like the two other variants of 
DSS: 

First, there is the signatory who generates a digital signature of the mes- 
sage M; then, there is the verifier who certifies the validity of the signa- 
ture. Every signatory disposes of a private key (for the generation of the 
signature) and of a public key (for the verification of the signature). For the 
two procedures (the generation and the verification of the signature) a digest 
of the message M will be extracted (by SHA-1, in general) in order to supply 
the document-support for the signature. 


Elliptic Curves Over Fom 


Our field of constants will be the field of residues Fgm = Fo[x]/(p(x)), where 
pia) = v™ +---+1 is an irreducible binary polynomial of degree m. For 
fixed m, all extensions of Fp of degree m are isomorphic. A (multiplicative) 
arithmetic which is easy to implement will be furnished by those p(a) which 
are trinomials or (if irreducible trinomials do not exist) by pentanomials. 
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Example Irreducible trinomials over Fy — some examples: 


gq i61 yl8 1, 
gil o? 1, 
255 2 1, 
77300 gp? 1, 
319 4 736 4 1, 
9399 26 1, 
gl 4 10 4 7, 
20 4 99 4 1, 
1023 4 at Cie. 1, 
1866 v 1, 
1596 fod 1, 
1980 33 1 
1994 gid 1 


o] 


There are no irreducible trinomials of degree m when m = 0 mod 8 - a 
surprising hostility of mathematics towards the classical formats in computer 
science. But irreducible pentanomials will exist, for all degrees m > 4. 


Trreducible pentanomials over Fz — some examples: 


160 4 gll7 4 92 4 op 4 1, 
oi pal 4+ a2 tat 1, 
7256 4 7155 4 92 4 op 4 1, 
2 4 4g toe 4-1, 
400 4 245 4 2 4 op 4 1, 
p12 4 gl 4 92 4 4 1, 
640 4 253 4 93 4 op 4 J, 
7800 4 7463 4 92 4 op 4 1, 
gp l024 4 7515 4 2 4 4 1 
1536 4 7881 4 92 4 ye 4 1 
1600 4 757 4 92 4 op 41, 
1920 4 767 4 2 4 op 4-1, 
gp POOO ig DEE oe a2 te ge hag: 


We shall now pass to the principal subject of this paragraph: the elliptic 


curves. 


Definition An elliptic curve! E(Fym) over Fym, defined by the couple 


(a,b) € (Fom)?, b # 0, is the set of solutions 


equation 


(X,Y) € (Fam)? of the 


Y?4+ XY = X°+ax7?+b 
plus one distinguished point O, which is meant to be the point at infinity. 


Notation #€(F2») = the number of points of E(F2m). 


" Our definition is a little bit restrictive: The condition b 4 0 happily eliminates 
the supersingular case which haunts the odd-prime-characteristic theory. 
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Remark (1) Pass to homogeneous coordinates: 

Y?Z4+ XYZ = X°+aX?Z+4 62? 

The projective curve which is defined by this equation is the correct object 
of the foregoing definition: 

In homogeneous coordinates (a projective point is nothing but a 1D linear 
subspace minus the origin) we have: 

© = [0,1,0] = the (0-dotted) Y-axis as a projective point . 

The finite points are then of the form [X,Y,1], where (X,Y) is a solution 
of the initial non-homogeneous equation.!? 


(2) #E(Fam) is even. 


By a celebrated theorem of Hasse’s we have the estimation 


27 41-272" < HE(Fom) < 27414 2V2™, 


For an elliptic curve over Fig this means: 10 < #€(Fieg) < 24 and for an 
elliptic curve over F256 this yields: 226 < #€(Fa256) < 288. 


Fundamental observation. E(Fam) bears the structure of an additive abelian 
group according to the following laws: 


1. O= the point at infinity, is the zero-element for the addition. 


2. The opposite point for the addition: —(X,Y)=(X,X4+Y) 


3. The sum of two distinct (finite) points (which are not opposite): 


(X1, V1), (X2, Ye) € E(Fam), X1 F Xo. 


(X1, Yi) + (X2, Yo) = (X3, Y3) 


with 
X3=dM+A4-X,4+ Xot+a, Y3 = \(X1 + X3) + X¥3 +N) 
where A= sity € Fom. 


4. Doubling a point (of order >2): 
(X1,¥1) € E(Fom), 2X1 #0. 
"2 For those who are not accustomed to homogeneous coordinates: In affine think- 


ing, the last coordinate is nothing but a common denominator for the first two 
coordinates. So, O is indeed at infinity — in direction of the y-axis. 
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2(X1, Yi) = (X3, Y3) 
with 


X3=M+A+a Y¥3 = X2+ (A+ 1)X3 


where A= X,+ 3 € Fom. 


Remark The (arithmetical) laws for the addition and the doubling of (finite) 
points on an elliptic curve have a precise geometrical meaning: 

First consider the case of two points P,; = (X1,Y1) and Py = (X2, Yo), 
which are distinct and not opposite (for the group operations defined above). 
P3; = P, + Py = (X3, Y3) can be constructed in the following way: 

Let £ be the straight line which goes through P, and through P . £ will 
intersect €(F2m) in a third point Pj. We will obtain: P; = —P§ (P3 will be 
the opposite of P§ for the group law on €(F2m)). 

Let us show that this geometric construction indeed yields the foregoing 
expressions: 

Let Y = AX + ¥ be the equation of £L: A= YE and y = ¥, + AX. 
A point P = (X,AX + y) of £ is at the same time onto E(Fom) if and only 
if (AX +7)? +AX?24+ 7X = X°+aX? +4 bd. Consider now the cubic equation 
XS 4 (N74 A+ aX? + 9X +97 +050. 

We know: The coefficient A? + \ + a is the sum of the three roots of this 
equation: X, + X]+ X3 = \?+ A+ a. This gives the expression for X3. For 
the ordinate of the third point of intersection of £ with E(Fgm) we get 


Y3 =AX3 +7 = A(X 4 X3) +N. 
Passing to the opposite point gives finally: Y3 = A(X, + X3) + X34+ V1. 


As to the doubling operation (P; = P2), let us remain in the logic of the 
foregoing arguments and replace the straight line £ which connects P; and 
P» by the tangent to E(Fam), at Py. 


Recall Let F(X,Y) = 0 be the equation of a plane curve, and let P, = 
(X1,Yi) be a simple point of the curve (S£(P,) and $£(P,) don’t vanish 
simultaneously). 
Then the equation of the tangent £ to the curve defined by F(X, Y) = 0 
at P, = (X1,Y1) is given by 
OF (PMX aay) oe (PY —¥%)=0. 
Specialize down to our particular curve €(Fam) : Y°?+XY+X%+aX7+b= 0. 
We have: ae = X?4Y, ae = X. This yields the equation of £: X,Y = 
(X?+Yi)X + X}. In case X 4 0, we obtain Y = AX +4, with A= X1 +}, 


7 = X?. With precisely the same argument as before we arrive at: 


130 2 Cryptography 


X3=M+A+a, Vg =AXZ+7=AX3+ Xj, 


Y3 = X3+ Ys = X74 (A4+1)X3, 
which was our claim ... 


Example An elliptic curve over Fig = Fo[x]/(x* + 2 +1). 
Recall: w = x is a generator of the multiplicative group F¥¢: 


W= 2, we = 72 +1, 
w? = 2, w =e +e, 
ws = 2, wl = 7+ x7+1, 
wt=a¢+1, wia=gB+e?+a, 
w= e+e, wl? = +e? +e41, 
w® =o + x, wis =o + 77241, 
woeP+e+l, wtoe? +1, 
wih =] 
Our elliptic curve E(Fi¢) will be defined by the equation Y? + XY = 


X3 4+ w4X?+41. And these are the 16 points on the curve: 
O (0, 1) (1,w°) (1,w?) 


(a? 09") (w3, w4) (w®, w3) (w®, wt) 
(w ,W ) (w®, wi) (w?, w4) (ca? ws?) 


(wut) wW!u8)  (w!2,0) (w!?,w"?) 


Let us carry out, as an example, an addition and a doubling of points: 
P, = (X1,N%) = (w®, w8) Py = (X2, Yo) = (w?, wu) 


dr — W+Yo _ wertw 
“ X1+Xo wotws 
6 3 4 


NXg =A? ++ Xp + Xo t+ a=W? +w+u* +0? +07 =1, 
Y3 = A(X1 + X3) + Xg4+ VY =w(w? +1) +1408 =o 


(Ce aaa 
2. P3 = (X3,¥3) = 2P: 
A=Xi + B= uh + | =, 
X3=NM4+At+asu® +u% +04 =", 
Y3 = Xf + (A+1)X3 = wl? + (w9 + Tw? =o. 


Exercises 


(1) Show that every elliptic curve E(F2m) admits a single (finite) point P of 
order 2 (i.e. such that 2P = QO). 

(2) Consider the elliptic curve €(Fig) of the foregoing example. 
Since #€(F 15) = 16, the order of a point P on E(Fy¢) is 1, 2, 4, 8 or 16. 
Determine the order of each of the points on the curve. Which are the 
subgroups of €(F1¢)? 
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(3) Another elliptic curve over Fig : €(Fig) = Y¥2 + XY = X3 43 
Let G = (w?,w®) 
(a) Show that G €E(Fje). 
(b) Compute 2G, 3G, 4G, 5G. 
(c) Show that 10G = (0,w’). 
(d) Show that E(Fi¢) is a cyclic group of 20 elements. 
(4) An elliptic curve over Fo5¢ = Fe[z]/(x° + 24 +43 + a4? +1). 


Notation: a = 2 mod (#8 + a+ + 23 + 27 +1). 
We know already: a is a generator for the multiplicative group Fxg. 


(a) Put w = a7 = 10011000. 


Verify that wt+tw+1=0. 

Deduce that the field Fig = F2[2]/(a* + 2 +1) can be identified with the 
subfield F2[w] = {0,w,w?,...,w!° = 1} of Fas6 (this means: the multiplicative 
group of 15th roots of unity in F3;,, augmented by 0, is stable under addition). 

We are interested in the elliptic curve defined by the same equation as in 
exercise (3), but now over Fos¢ : E(Fass) = Y2 + XY = X34+u3 

Our goal will be to find #€(Fos6), ie. the order of the group of Fos¢- 
rational points on the curve (note that we are allowed to solve our given 
elliptic equation over any field k which is an extension of Fig). 


(b) Let P = (a,a®%) = (00000010, 11111110) € Fag. 
Show that P € E (Fase). 

(c) Compute 2P, 4P, 8P, 16P, 32P, 64P, 128P, 256P. 

(d) Show that #E (Fase) = 280. 


(Hint: €(Fic) is a subgroup of E(Fe56); hence #E(F256) is divisible by 20. On the 
other hand, 226 < #€(F256) < 288, by the Hasse estimation. Conclude now by 
means of (c)). 


Finding Elliptic Curves with Prescribed Order: The Complex 
Multiplication Elliptic Curve Generation Method (Lay—Zimmer 
method) 


We want to discuss (and to solve) the following highly non-trivial problem: 
The input data: 


(1) A field of scalars Fom = F9[x]/(p(a)) , where p(x) = 7 +---+1 is an 
irreducible binary pentanomial (or even a trinomial). 

(2) A large prime number n — which will determine the magnitude of the 
cyclic range for our signature arithmetic, i.e. which will be the order of a 
(cyclic) subgroup of some elliptic curve E(Fam). 


The output data: 
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The equation Y*+XY = X3+5, b € F5m of an elliptic curve €(Fym) 
over Fym such that N = #€(Fos6) is nearly prime! in the following sense: 
N = 2"-n, r > 2 (actually, we only are interested in N = 4n). 


In the sequel, we shall describe the algorithmic steps which finally realize 
our programme. It should be pointed out that the underlying mathematical 
structures are rich and rather complicated (we are in the heart of Algebraic 
Number Theory) — so, an exhaustive presentation would be largely beyond 
the scope of this book. 

Our goal is to make the algorithms as transparent as possible. Nevertheless, 
at the end of this section, we shall try to give a brief sketch of the theoretical 
background. But let us resume: 

Starting with a scalar field of 2” elements and a prescribed group or- 
der N = 4n, we are |in search of the non-zero constant term 6] of the elliptic 
equation Y? ++ XY = X?+5 which will have the desired number of solutions. 


We will have to solve three algorithmic problems: 


(1) Find a CM discriminant D for 2”. 
[D is the arithmetical link between the scalar-field degree and the desired 
group order NV 

(2) Compute the reduced class polynomial wp(t) of D. 

(3) Find a particular root @ (in Fgm) of w(t) = wp(t) mod 2; then the con- 
stant term b of our elliptic equation will be either equal to @ or to 6° 
(depending on D = 0 mod 8 or not). 


So, our first goal is to introduce the parameter D which controls the transit 
from our input data to our output data. 


CM Discriminants 
Let € = €(Fom) be an elliptic curve over Fgm, of order N. Then Z = 2™*? — 


(2”+1—N)? is positive, by the Hasse Theorem. There is a unique factorization 
Z = DV? with D (odd and) square-free. We can write 


(x) 242 = W24 DV? 


(xx) N= 2™+14W 


We shall say that € = E(F2m) has complex multiplication by D (better: by 
VED). 

D will be called a CM discriminant for 2”. 
18 We define near primality as simply as needed; whenever N = 0 mod 4, we are 


allowed to search for elliptic equations without quadratic term in X, i.e. satisfying 
a=0. 
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Exercise Show that we have necessarily: D = 7 mod 8. 


In order to well understand the following algorithmic steps (CM discrim- 
inant test and class number computation), we should briefly recall some ele- 
mentary facts from Algebraic Number Theory. 


Ideal Classes in Imaginary Quadratic Fields — a Summary 


We shall restrict to the case which we actually need. So, let D be a positive 
square-free integer, D = 7 mod 8, and put d = —D. Note that d= 1 mod 4.4 

K = Q(vVd) = {r+sVd:r,s€Q} CC _ is the imaginary quadratic field 
defined by adjunction of Vd to Q. 

Put w = 1ivd = 1HiVD 

R=Zlw] = {n+mw:n,me€ Z} is the ring of (algebraic) integers in K 
(consisting of the elements of K which satisfy an integer quadratic equation 
with leading coefficient equal to 1). 

Ris a lattice in the complex plane, freely Z-generated by 1 and w = tae 
Notation R= (1,w)z. 

We are interested in the sublattices of R which are ideals of R (i.e. which 
are stable under multiplication with complex numbers of the form n + mw, 
n,m € Z). 


Proposition A sublattice I C R is an ideal of R <> there exist 
a,b,c,y € Z such that b? — 4ac = d,a > 0,y > 0, and I = y(a, btivD ) = 


{nya + my tvB in, m€ Z}. 


The arithmetical complexity of K (equivalently: of R) is roughly encoded 
in the classgroup Cl(K) of K = Q(Vd). So we have to speak about fractional 
ideals. 

A fractional ideal J C K is an R-submodule of K of the form J = +1, 
with some fixed n € N, and some ideal IJ of R (you should consider J as 


a refinement to the order n of the lattice I). Hence, as a consequence of 


b+iVD 


proposition 1, a fractional ideal J is of the form J = (a, 3 


as in Proposition 2.4.5, and some fixed positive y € Q. 

Fractional ideals multiply in a straightforward way: 

IJ = {all finite sums of products with one factor in J and one factor 
in J}. In particular, consider, for a fractional ideal J, its conjugate o(J) = 
{€:€€ J}. Then o(J)J = Jo(J) = (N(J)) = the principal (fractional) ideal 


b+i/D 
2 


)z, with a, b,c 


generated by the norm of J (which is, for J = y(a, 
rational number 72a). 


)z, equal to the 


4 This singles out one of two types of imaginary quadratic fields, but will not 
substantially touch upon the generality of most of our results. 


134 2 Cryptography 


As a consequence, we get: nnd) = (1), ie. every fractional ideal J 
admits a multiplicative inverse J~! = wo): Thus the set of all non- 


zero fractional ideals of kK = Q(vd) is an abelian group under multiplication 
(of fractional ideals). This group is much too large in order to appropriately 
describe the type of the particular number field K = Q(Vd). 

Finally, the good object is the quotient group modulo the subgroup of 
principal (fractional) ideals — the classgroup Cl(K) of K = Q(Vd). We shall 
see in a moment that Cl(K) is a finite group. Its order, the classnumber h(K) 
of K, is an important invariant of the number field Kk. 

So far, we did not substantially make use of the imaginary type of our 
quadratic number field. But now we shall proceed to give a “minimal para- 
metrization” for the classgroup Cl(/c), where it will play an essential role. 


Proposition Every ideal class of an imaginary quadratic field with discrim- 
b+iVD) with 


inant!’ d = —D admits a representative of the form I = (1, 

a,b € Z,a > 0, 4ac — b? = D (and appropriate c € Z). 

The ideal class [I] is completely determined by 7 = btivD 

upper half-plane H = {z € C : Im(z) > O}. 

btiVD 
2a 


, which lies in the 


and 7/ = u+ivD of H represent the same ideal class 

ie 98 the group of integer 2 x 2 

<= there exists A = ( a € SL2(Z) = matrices with determinant 
2 equal to 1 


Two points T = 


at+p 


with 7’ = a 
T+ 


Now consider the set of all maps 


A:H3z+-A(z)= oat €H with A= 5 ) € SLo(Z).'® 


We get the modular group T. Since the matrix A determines the transfor- 
mation A up to a sign, we may identify: 


T = PSL9(Z) = SL2(Z)/{+1}. 


Note that I is generated by the two “natural” transformations T and S 
induced by the matrices 


1 1 0 -1)\. 
r=(5 4) and $= ({ 5.) in Sta(@). 


Geometrically, we have T(z) = z+1 and S(z) = —4, ie. T is a horizontal 
translation by a unit, and S is a reflection on the unit-circle, followed by a 
reflection on the imaginary axis. Observe that the order of S as a matrix is 4, 
whereas the order of S as a mapping is 2. 

According to proposition 2, an ideal class of K = Q(Vd) determines an 
orbit for the action of I onto H. Thus we are interested in canonical repre- 
sentatives for orbits. 


' Recall: We focus on the case d= 1 mod 4. 
16 We do not notationally distinguish the matrix from the induced transformation. 
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The fundamental region. 


The fundamental region. F of the modular group T is defined as follows: 
F = {ze€C;| Re(z) |< 5,| 2 |>1}- 


Proposition F is a fundamental region for the (action of the) modular group 
T in the following sense: 


(a) Every z € H is modF equivalent to a point z'’ € H: there is A € T: 
z' = A(z) EF. 

(b) Let z and z' be two distinct points of F. Then z = 2’ modT <=> 
z and z' lie on the boundary of F; more precisely: one of the following 
alternatives hods: 


(i) | Re(z) |=| Re(z’) |= §, and 2’ =z+ 
(ti) | z |=| 2’ |= 1, and 2’ = —2. 


T 
— 


Hence: Every z € H admits modulo T a unique representative 7 € F — with 
Re(r) > 0 tf | Re(r) |= $ or |7 |=1. 
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Consequence (Reduced ideals) In every ideal class of K = Q(/—D) 
there is a unique ideal I = (a, eyD) such that 


(1) a,b € Z,a>0, b? + D = 4ac with appropriate c € Z, 
(2) |b|<a@<ce, and b > 0 whenever either | b |= a or a=c. 


Note that for 7 = bev we have: Re(r) = £ and | 7 |?= ¢ 


j= (a, tt) is said to be reduced (whenever (1) and (2) hold). 
Zz 

Now it is easy to compute the classgroup Cl(K): 
You only have to determine all triples (a, b,c) satisfying (1) and (2). 

Note that 6? + D = 4ac and | b |< a < cyield | b|<a< / 8; thus there 
is only a finite number of possibilities. 
Example D = 163 (a little bit non-thematic: D # 7 mod 8, but still —D = 
1 mod 4). 

188 = 7.37, this gives 1,3,5,7 as positive candidates for b (which has to 


be odd: 6? + 163 = 0 mod 4). Now, #4163 = 41, 43, 47,53 for b = 1,3,5,7. 
Thus only (a,b,c) = (1,1, 41) is possible: Cl(Q(./—163)) = {1}. 


[Remark What can we say about the second type of imaginary quadratic 


fields K = Q(V/d), with square-free d < 0 and d = 2,3 mod 4? 

Here, the field discriminant is equal to 4d = —D. 

We insist: For Q(./—1), the field discriminant is —4, for Q(/—2), the field 
discriminant is —8, for Q(/—5), the field discriminant is —20, etc. 

The classgroup computation is relative to D = 0 mod 4 (since normal 
forms for fractional ideals are relative to D). 


Example D= 20 (corresponding to K = Q(/—5)). 
We search for (a, b,c) with b?-+20 = 4ac (hence b will be even), | b |< a <, 
and b > 0 whenever either | b |= a or a=c. 


Now, |b|<a< \/ 2 implies b = 0 or b = 2; this gives 


Thus h(Q(v=8)) = 2. 
Exercises 
(1) Recall the two standard generators T = ( :) ands = (! a 


of SLo(Z). Put R= & ca R, = -R. 
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(a) Check that R = ST and that R(z) = 
(b) Check that T = SR. 


(c) Show that S* = R} = (5 ic) Hence: S? = R? = Id as mappings. 


= (as a mapping). 


(2) Show that h(Q(./—7)) = 1. 
(3) Find the seven reduced representatives in the classgroup of Q(./—71). 


(1, 1,18), 
Answer (a,b,c) = ey 
(4, +3,5). 


(4) Find, for positive square-free D = 7 mod 8, the minimal values such that 


h(Q(./—D)) = 2,3, 4, 5,6, 7. 


| 2 for D= 15, 
3 for D=23, 

4 for D=39, 

Answer h(Q(/—D)) = 5 for D=47 
6 for D= 63, 

7 for D=7l. 


Remark Let hp = h(Q(V—D)), where —D is the discriminant of the imagi- 
nary quadratic field. Then 


km l0sthe) _ 
im =— = = 
Doo log(V D) 


Thus, for large D, h = hp is roughly of the magnitude of VD. 


Historically, the classification of fractional ideals in imaginary quadratic 
number fields has been imbedded in the classification of certain binary 
quadratic forms. 

So, consider (positive definite) homogeneous quadratic forms 


b 
2 2 a 7. xX 
q(X,Y) =aX*+bXY+cY?=(X Y) (; ?) @) 

with integer coefficients a,b,c € Z,a > 0, and such that D = 4ac — b? is 
positive and square-free. We shall say that g(X,Y) represents the integer 
u€E€Z <= u = qa,y) for appropriate integers x,y € Z. Note that 
q(X,Y) always represents the coefficient a (for (x,y) = (1,0)). 

The quadratic form q/(X,Y) = a'X2+b'XY +cY? is said to be equivalent 
to 

(X,Y) =aX?+bXY 4+ cY? whenever there is U = & a € SL2(Z) 
with 7 (X,Y) = q(aX + BY, yX + oY) (invertible integer change of coordi- 
nates). 
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’ b! 
This simply means that Fu 2) Sa ( 


b 
2 )U. 
2 e 
We get an equivalence relation for (positive definite) quadratic forms (with 
negative square-free discriminant —D.'") 
Equivalent quadratic forms g(X, Y) and q'(X,Y) represent the same inte- 
gers: 


/ 
u=q(2,y) SS u=_d (a, y’) with (3) =U! ( 


eo) 
NIe & 


x U has the same 
y meaning as above. 


Now, the equivalence relation for quadratic forms with discriminant —D 
is “the same” as the equivalence relation for (normalized) fractional ideals of 
K = Q(V-D). More precisely: 


Exercise 


rot b 
Consider {| §, 2 | =U? (; 2 ) U, withU = i B ) € SL(Z) and 
ge x3 ¢ y 39 
positive square-free D = 4ac — b?, a > 0. 
Put 7 = 2tivp — UtiVD Then: 7! = ot+8 
2a 2a ** . 


T Sap 
yTta 


Thus a reduced triple (a,b,c) (with a,b,c € Z,a > 0, b? + D = 4ac and 
such that | b |< @ < c— with b > 0 whenever either | 6 |= a or a = c) has two 
meanings: 

It (minimally) represents 


— The ideal class [I] = [(a, 24“2)z] of K = Q(V—D) 
— The (class of the) quadratic form q(X,Y) = aX? + bXY +cY?. 
In the sequel, it will be convenient to work with integer symmetric matrices 
G - such that A > 0; the determinant D = AC — B? is supposed to 
be positive and square-free (we shall always have D = 7 mod 8). 

Thus we consider (positive definite) quadratic forms Q(X,Y) = AX? + 
2BXY + CY? with discriminant equal to —4D. 

Note that this “arithmetical zoom to the order 4” is not completely harm- 
less. 

First, trivially, Q(/—4D) = Q(V—D). 

But how can we get a faithful parametrization of the classgroup 
Cl(Q(/—D))) in terms of (appropriately) reduced matrices [A,B,C]!® = 


(sc) 


Let us explain our problem by a simple 


17 This definition of the quadratic-form discriminant aims to harmonize with field 
theory. 
18 This is a definition of the bracket-triple. 
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Example D=7 

We already know that Cl(Q(/—7))) = {1} (there is a single reduced 
(a,b,c) = (1,1,2)). But there are two non-SL2(Z)-equivalent reduced 
quadratic forms with discriminant —4 - 7: 


(1, 0, 7] 
[A,B,C] = { (2, 1,4] 


Reduction via the fundamental-region argument: We have to search for 
all [A, B, C] such that D+ B? = AC, |2B|< A< Cand B>Oif|2B|=A 


or A=C. 


We note that we will have to sift out “redundant” representatives [A, B, C] 
by an extra-condition. This finally gives the following. 


Proposition Let —D =1 mod 4. 

Then Cl(Q(./—D))) can be identified with the set of symmetric matrices 
[A,B,C] such that D+ B? = AC, and which are strongly reduced in the 
following sense: 


(1) |2B |< A<CandB>0if|2B\|=A or A=C. 
(2) ged [A,2B,C] = 1 (i.e. either A or C has to be odd). 


The ambitious reader is invited to prove the statement as an exercise. 
But let us look at an 


Example D = 71. We get for Cl(Q(./—71))) the following alternative para- 


metrizations: 


[L057], (1,1, 18) 
[3, +1, 24], (2, +1,9), 
BO = (8, +1, 9], (a,b, ¢) = (3, £1, 6), 
(5, +2, 15] (4, +3, 5). 


For odd B = Bb, the transcription from [A,B,C] to (a,b,c) is easy to 
understand. But for even B? How can we get (4,+3,5) from [5,+2, 15]? 

Start with Q(X,Y) = 5X? +4XY + 15Y? and substitute X —+ 42*, 
Yr-> ~~ 

This gives ¢(X,Y) = 4X? — 5XY + 6Y°. (a,b,c) = (4,—5,6) is not yet 
reduced. 

(4, —5,6) +> (4,3, 5) is the desired reduction: b > b mod 2a corresponds 


to 
ey oa ee a a 2ko-tb 
k 1/\8 O 1) \ ath ak? + bk +c 


for appropriate k € Z. This reads, in terms of the T-action on H: qh (Paw) = 
b+ivD | 5 — b+2katiVD By the way, S(LtivB) — =b+4iVvD _ which yields the 
reduction step (a, b,c) += (c, —b, a). 


oO NI 
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Finally note that our substitution X | = =. ,Yt Ss corresponds to 
B+iV/D , ttl  b+iVvD 
T= t i — ; 
A —T+1 2a 
with 
_ (A-2B+C) AS) 
7 4 ae ae 


This finishes our summary on ideal classes of (certain) imaginary quadratic 
fields. 


Now we are ready to treat our first problem: Let m € N be a fixed field 
degree. 

How can we decide whether a given positive square-free integer D (with 
D =7 mod 8) is a CM discriminant for 2”? 

The question is: Does the diophantine equation 2+? = X?+ DY? admit a 
primitive solution (x,y) € Z? (i.e. with coprime coordinates — note that x then 
must be odd)? (in another terminology: does the quadratic form Q(X, Y) = 
X? + DY? properly represent the integer 2™*??) 

Before presenting the compliance test for D, we have to resolve some minor 
technical (and conceptual) problems. 


Exercises 


(1) (Finding square roots modulo a power of 2). 

Let d= 1 mod 8. 

Consider the congruence X? = d mod 2”, n > 3. 

Show that it admits exactly four solutions; more precisely: The four solu- 
tions are of the form {t,2"~! — t,2”-! + ¢,2” —t} with 1 <t < 2”-?. 
Hint You should proceed by induction on n. Note that the four solutions 
on level n give eight candidates on level n+1, which split into two families: 
{t,2” —#,2" + t,2"*1 — t} and {2-1 — ¢,2"-1+#,27 + 2"-1 —4,2"41 — 
2”-1 + t}. Only one of them is acceptable. 

Find, following the “exclusion method” proposed by exercise (1), the four 
solutions of 


— 
i) 
YS 


X?2+471 =0 mod 1024 


[ Answer: 235, 277, 747, 789. 
(3) (Square roots modulo a power of 2: The algorithm.) 


Let d= 1 mod 8. 

Consider the congruence X? = d mod 2”, n > 3. 

We search b, 1 < b < 2”~?, such that b? = dmod 2” (note that b is 
unique!). 
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Show that the following algorithm yields b. 


d = bn_16n—-2°++6169 (binary notation) 


at ea ae (binary notation — 6 will be constructed via t 

me kai "0 step-by-step, i.e. from the tail to the head) 
(binary notation — u means ¢”, the approximation 
of d = b? according to the widening of the arith- 
metical window in the spirit of the argument of 
exercise (1)) 


U = Un—1Un—2°°* U1U0 


. Initialize: t =1, u=1. 


Note that 626,69 = 001 and 747) = 01 


. tr >b: For 2<j<n-2 do: 
If Uj+1 x Oj41; set Ty 1. 
If 7 < 2, set u = (ut 2941¢ — 27/) mod 2”. 
else set u = (u+ 2/*1t) mod 2”. 
. Normalization: If t,_2 = 0, set b = t, else set b = 2"—! —t. 


(5) 


(6) 
(a) 


Recall the hint to exercise (1): Whenever X? = d mod 2/*1 has exactly 


the solutions x = +t mod 2/, then X? = dmod 2/*? has either x = 
+t mod 2/+!, or x = +(2/ + t) mod 2/*" as its only solutions. 

Now, ?—d=2+!. ie ke Z, means 53 = Uj, bj—-1 = Uj-1,°°° 

k even => Oj41 = Uj41 k odd => Oj41 x Uj+1- 


But this parity alternative on k precisely corresponds to the alternative 


on the 2-parametrization mod2/*!. 


Following the preceding algorithm, find the minimal positive solution 6 of: 
(a) X? = 73 mod 256 

(b) X? = 73 mod 65536 

b = 29 in the first case, and b = 4253 in the second case.| 


Find, by means of the algorithm of exercise (3), the minimal positive 
integer b such that b? + 71 = 0 mod 23°. 


Answer b = 17173.| 
Let Q(X, Y) = AX? + 2BXY + CY? be reduced: | 2B |< A<C. 


Show that Q(z,y) > (A—2|B|+C)-min(x?,y?) , and hence 


Q(z,y) > A-2|B|+C for cy 40. 
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(b) Deduce from (a) that the outer coefficients A and C of Q(X,Y) give the 
minimum values properly represented by any equivalent form. 


(7) Show the following assertion: If [A, B, C] is equivalent to the reduced form 
[A’, B’, C’], then [A, —B, C] is equivalent to the reduced form [A’, —B’, C’]. 
Consequence: If B’ = 0 or necessarily positive (B > 0 if | 2B |= A or 
A=C), then [A, B,C] is equivalent to [A, —B, C]. 

Hint Every reduction is a chain of alternating actions of S = G 4) 
and T* = & : for appropriate k € Z. Show that the assertion is 
true for every step. 

(8) Suppose that Q(X, Y) = X?2+ DY? properly represents 2”? (i.e. with 
relatively prime coordinates x and y). 

Find an equivalent quadratic form Q!(X,Y) = 2™+?X2 +2BXY + CY? 
(which trivially represents 2’"*?). 
Hint Write 2"+2 = 2? + Dy?, (x,y) € Z? -— where x and y are 
coprime. So, rx + sy = 1 for appropriate r,s € Z. Then compute 
uy 1 0 Zz -Ss 
-—s 0 D ae ae 
(9) Show the following equivalence: 


4X? +2BXY +CY? 

(D + B? = 4C) properly 
represents 2””*? by (2, y), 
with y = 0 mod 2. 

Sketch of proof. First note that AX? +2XY + CY? = 4[((AX+BY)?+ 
DY? |, 

Thus 4X?+2BXY+CY? =2™+? <> (2X + BY)? + D(X)? =2"*?, 
Then, you have to find the argument for the correctness of the transfor- 


X?4 DY? properly represents 2+? <> 


mations ; 
€=227+ BE x= 5(€ — Bn) 
n= 4 y = 2n 


2 
(10) D>23,9 D=7mod8,m>1. 


Let Q(X, Y) = 2™?t?X? 4 2BXY + CY? be a quadratic form with “leading 
coefficient” A= 2™+?. So, D+ B? =2™+2C, 


(a) Show that there are — up to equivalence — only two couples of possibilities 
for Q(X, Y): [pre +B,, C1] and rere: +Bo, C4]. 

(b) If X? + DY? properly represents 2”’+?, then one of the couples reduces 
to [1,0, D], whereas the other couple reduces to [4, +B, C]. 


19 1) = 7,15 present some particularities. . . 
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Help 


1 0) (22 BY) /1 k\ (2m? BY er 
(a) € ee ale i - Cy wath OB 


k-2™+? + B, ie. B' = Bmod2™*?. Hence, appropriately choosing k, 
we get 0 < B’ < 2™*?. But there are precisely four distinct roots of 
X? + D=0 mod 2'”*? (cf exercise (1)). 

(b) First note that reduction respects sign-alternatives in the middle coeffi- 
cient B; so, whenever X? + DY? properly represents 2”*?, one of the 
couples of (a) must reduce to [1,0, D]. Now use exercise (9) — also look at 


I 


the hints for the next group of exercises. . . 


The following algorithm will 


(1) Decide, whether the diophantine equation 2+? = X? + DY? admits a 
primitive solution; 
(2) Provide, in the affirmative case, the (odd) z-value of a solution. 


The idea of the algorithm is simple: Start with the quadratic form 
Q(X, Y) = 2™ 2X? + 2BXY + CY? (such that D+ B? = 2*C) and find 
the equivalent reduced form. The answer of the algorithm is positive, whenever 
we arrive at the situation described in exercise (10); else, it is negative. 

Let us first explain the basic reduction step of the algorithm: 

It transforms a (positive definite) matrix 


a(t) 


(with A > 0) into an equivalent matrix 


A OB’ 
s=(% a 


which will be “closer to reduced form”. More precisely, let 


A B 
s-(5 c) 


and put 6 = [2+ 4]?° 


1 
2 


Then 


20 This is an asymmetrical round. 
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We have: A’ = C, B'= -B4+6C, C’ = A-26B4+ 86°C. 
Since 6 << B+4<d5+1 ie. 6C < B+4C < 6C+C, we get | 2B’ |< A’. 
For A’ < C’, reduced form is achieved. Otherwise, we have to continue, etc. 
How do we approach reducedness? 

Since D = A’C'—B”? = CC’—B”, we have D < CC’ ,ie. & < oe Hence: 
A! < C’ whenever C < VD. Now, at every iteration of the reduction step, the 
value of A decreases (we only iterate the reduction step in case C < A — but 
then A’ = C). So, after a finite number of steps, A’ = C' will be sufficiently 
small; this means that we dispose of the desired inequality C < VD, and we 
can stop. 


Proposition (Testing for CM discriminants) 

The following algorithm decides, for a square-free positive integer D > 23, 
D = 7mod8 and a field degree m > 1, whether D is a CM discriminant 
for 2™. 

In the affirmative case, it provides an (odd) integer x such that 2™*? = 
x? + Dy? (for some y € Z). 

Otherwise, the message will be “Not a CM discriminant”. 


Algorithm (1) Compute B such that B? = —D mod 2™*? (see exercises 
(1) to (5)). 
(2) Initialize A=2™*?, C= a S= - a and (x,y) = (1,0). 


BC 
(3) Iterate the reduction step until|2B|<A<C: 


/ / 
s= (4 e)—s=(% oy) = UsU, 


BC B! CO! 
with d= [244], ce 7 ) € 5Eal2) 


Compute (x',y’) = (62 + y, —2). 
Reset the target-variables to source-variables. 
(4) The final decision: 
If A=1, then output x and stop. 
If A=4 and y = 2n is even, then output 2x + Bn and stop. 
Otherwise, output “Not a CM discriminant”. 


Proof Everything has been treated in previous exercises. The exclusion of 
D =7,15 is explained in the next set of exercises. 

Note that we can get rid of this exclusion by a modification of the algorithm 
which stems from the following observation (cf the hints to the next exercises): 

Whenever we initialize with a middle-term B that corresponds to an odd 
outer coefficient C — and this is one alternative for the minimal positive choice 
of B — then the alternative in the final decision disappears, and we can treat 
all D=7 mod 8 — without any exclusion. 
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Exercises 


(1) 


(2) 


(5) 


Show that our reduction algorithm always provides a primitive solution 
(x,y) of the final reduced equation Q(X,Y) = 2™*? (primitive: x and y 
are relatively prime). 

D=7. 

Note that X? + 7Y? = 64 admits (1,3) as a primitive solution. 

Now consider Qi(X,Y) = 64X72 4+22XY+2Y? and Q2(X,Y) = 
64X? + 42XY + 7Y%. 

(a) Show that Q,(X,Y) reduces to 2X2 +2XY + 4Y?. 

(b) Show that Q2(X,Y) reduces to X? + 7Y?. 

Thus we have to be cautious: (a) will provoke a negative (hence false) 
output of our test algorithm. 

D=7 m>21. 

Show that X? + 7Y? = 2™*? always admits a primitive solution (2, y). 


| Hint 2+? X27 19BXY+CY*%, with 7+ B? = 2™+?.C, reduces to either 

(1,0, 7] or [2,1, 4]. 

The alternative depends on the parity of the outer coefficient C’. 

Show: Replacing B by 2”*! — B changes the parity of oA 

D=15. 

(a) Show that we get four reduced quadratic forms AX? +2BXY + CY? 
and that the classnumber h(Q(./—15)) = 2. 

(b) Show that X? + 15Y? properly represents 2+ = 16 and 2° = 64, but 
not 2° = 32 and 2” = 128. 

(c) Now consider the case 2+? = 28 = 256. 
Start with Qi(X,Y) = 256X2+78XY+6Y2 and Qo(X,Y) = 
256X? + 178XY + 31Y?. 
Show that Qi(X,Y) is equivalent to 4X? + 2XY 4+ 4Y?and that 
Q2(X,Y) is equivalent to X? + 15Y?. 
Caution The reduction of Q\(X,Y) to4X?+2XY+4Y? transforms 
(1,0) to the primitive solution (6,—7), which will cause a negative an- 
swer of our compliance test. Clearly, a transposition X <-> Y would 
resolve our problem ... 

(d) Finally, consider the case 2”*? = 29 = 512. 
Start with Qi(X,Y) = 512X?+78XY+43Y? and Q2(X,Y) = 
512X? 4+ 434XY 4+ 92Y?. 
Show that Q(X, Y) is equivalent to 3X? + 5Y?and that Qo(X,Y) is 
equivalent to 2X2 +2XY +4 8Y?. 
Thus D = 15 is not a CM discriminant for 27 = 128. 

D=15. m>1. 

(a) Show that X? + 15Y? properly represents 2+? <= >  m is even. 

(b) Show that 2X?+2XY+8Y? properly represents 2". <=> mis 
odd. 
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Hint First note that starting our test with Q(X, Y) = 2™?X?42BXY+ 
CY? (such that D+ B? = 2™+?.C), we always may assume B = t or 
B=2™+! _¢, with 0 < t <2”. For one of these cases, C is even, for the 
other C is odd. Only the odd case can reduce to X? + DY. 

For D = 15 we get: The odd case reduces to either [1,0,15] or [3,0, 5). 

The even case reduces to either [2, 1,8] or [4, 1, 4]. 

(a) You should eliminate undesirable constellations by arguments mod3. 

(b) 2X2 +2XY +8Y? =2™+?2 as (2X +Y)? + 15Y? = gms] 

D = 23. 

(a) Show that D = 23 is a CM discriminant for 2? = 8, but not a CM 
discriminant for 24 = 16. 


i) 2° = 32? 
(b) Is D = 23 a CM discriminant for (i) 2° = 32% 


(ii) 2° = 64? 


[ Answer (b)(i) no (ii) yes. 

Note that (i) gives an interesting negative answer: X2+23Y? = 128 admits 
the solution (x, y) = (6,2). But 128X? + 38XY + 3Y? reduces to 3X? — 
2XY+8Y? (and 128X74+90XY+16Y? reduces to 2X?+2XY+12Y"). 
Thus there is no primitive solution of X? + 23Y? = 128. 

(ii) 256X? + 90XY + 8Y? reduces to 4X? + 2XY + 6Y%, which admits 
the solution (x,y) = (5, —6). This gives the solution (x,y) = (7,—3) for 
X74 23Y? = 256. 

D=71. 

Show, following the reduction method of our compliance test, that D = 71 
is not a CM discriminant for 2° = 256. 


Solution: Let us be generous: We shall reduce both Qi(X, Y) = 1024X?+ 
470XY +54Y?2 and Qo(X,Y) = 1024X?2 + 554XY 4+ 75Y?. 
In the first case, we obtain 8X2 +6XY + 10Y?. 


In the second case, we obtain 8X? + 2XY 4+ 9Y?. 
Decide whether D = 71 is a CM discriminant for 2!°. 


[ Solution: The reduction steps of our algorithm: 
(262144, 17173, 1125] -— [1125, —298, 79] +> [79, —18, 5] +> [5, —2, 15] 


Hence D = 71 is not a CM discriminant for 2!°. 


Let us sum up: 
The essential mathematical content of this paragraph is the following: 
First, the couple (N, 2”) which describes the desired field degree m as well 


as the desired number of points N on the elliptic curve to be designed, intro- 
duces the integer D, which is interpreted as the (opposite of the) discriminant 
of an imaginary quadratic number field K = Q(V—D). 
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Then, the h = h(K) ideal classes of K = Q(W—D) define h distinct 


lattices of the complex plane, parameterised by h numbers 7, = btivP 


a) 


Th = bativD in the fundamental region (of the modular group). 
It is precisely the appearance of this “lattice-shaped” geometry which will 
help us to realize our programme. 
Note that the quadratic forms are only “technical slaves”, without con- 


ceptual importance. 


Finding a Nearly Prime Order 


Having fixed the desired field degree m > 1 and a lower bound for the prime 
order n of the cyclic subgroup to support our signature arithmetic, we have 
to find 


a square-free positive integer D = 7 mod 8 and a sufficiently large prime n, 
such that N = 4n is the order of an elliptic curve over Fym with complex 
multiplication by D. 


We have to proceed as follows: 


1. Choose a square-free positive D = 7 mod 8. 

2. Compute h = h(Q(/—-D). 

3. If m does not divide h, return to 1. 

4. Test whether D is a CM discriminant for 2”. If not, return to 1. Otherwise 
the result of the compliance test is x € Z. 

5. The possibilities for N are 2™+1+<2. Now the question is whether N = 4n, 
with n prime (and sufficiently large). If the corresponding order is o.k., 
output (D,4n). Else return to 1. 


Example Fix m = 155. 
For D = 942679 we get h = 620. So, m correctly divides h. 
Our CM discriminant test yields: 2'57 = 2? + Dy?, with 
X = 229529878683046820398181, 
y = —371360755031779037497. 


Finally, we get 


25 41—2¢=An, 


where n is the prime n=11417981541647679048466230373126290329356873447. 
Thus there is an elliptic curve over F155 of order N = 4n having complex 
multiplication by D. 
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After these rather lengthy preliminaries on CM discriminants, our next goal is 
a Shameless and direct algorithmic attack of our main problem: to find quickly 
the constant term 6 of the desired elliptic equation. 

First let us have a look at the tip of the modular function iceberg: Consider 


anya Ve (37?7-5)/2 4. (39? a spp te BOT og VE As 


Let D be a positive square-free integer, D = 7 mod 8, and let [A, B,C] be 
an integer symmetric matrix with determinant D = AC — B?. 
Put 6 = exp( 44+ =VD+Bi Tt) 

Let 

f(A, B,C) = 0-2» SGP) 


f; (A, B, C) = 0-2 = mm 


) 
f,(A, B,C) = V2. 0m . 74) 


Observation | @ |< e~*¥3/2 = 0.0658287. 


Thus we control perfectly the speed of convergence of the power series F'(z) 
used in computing the numbers f,(A, B,C), & = 0,1, 2. 

If D and [A,B,C] are as above, then we define the class invariant 
c[A, B, C] of [A, B, C] as follows: 


Aye (-1 A. eH B(A-C+4°0) (4, B,C) AC odd 
c[A, B, Clo = oie en B(A+2C—AC*) . (4, B,C) C even 
1 ene O-40). (ABC) A even 


_ fclA, B,C]o for D#0 mod 3; 
c[A, B,C] = { (c[A,B,C]o)? for D=0mod3. 


Let h = h(Q(V—D)) be the classnumber of K = Q(/V—D), and let [Ai, Bi, C1] 
..[An, Bn, Ch] be the h reduced symmetric matrices which represent faith- 

fully the ideal classes. 

Definition The reduced class polynomial for D 


h 


wo(t) = | [(@- ¢[A;, B;, Cj). 


j=l 


Proposition wp(t) € Zit] (i.e. wp(t) has integer coefficients). 
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Note that this result facilitates the computation of wp(t): You only need a 
rough approximation of wp(t), which uniquely rounds to an integer-coefficient 
result. 


Example D = 71. 
Recall the seven reduced symmetric matrices with determinant D = 71: 
(1,0, 7], [3, 41,24], [8,+1,9], [5,+2, 15], 


1 
c(1, 0, 7| = —=fo(1, 0, er 


f2 
c[3, 1,24) = ee, (3,1, 24) 
gto —3 V2 1\9, 4, 5) 
c[3, —1, 24] = te-we (3, —1, 24) 
’ ’ a V2 1\e¥%,> ’ 5) 


237i 


1 
c[8, 1,9] = Va =< f2(8, i;9); 


1 —237i 
c[8, —1,9] = —~-e~2" f2(8, —1,9) 
J2 


1 Bari 
c[5, 2, 15] = ——~e 1 (5, 2, 15), 
J/2 


1 57i 
c[5, —2, 15] = -—e7 fy(5, —2, 15). 
J2 


We obtain: 


c[1, 0, 7] = 2.13060682983889533005591468688942503. . . 
c[3, 1,24 0.95969178530567025250797047645507504... 
+0.349160710012696547998553 16293926907... 2 
0.95969178530567025250797047645507504... 
—0.34916071001269654799855316293926907 . ..2 
—0.7561356880400178905356401098531772... 
+0.0737508631630889005240764944567675 . ..2 
c[8, —1,9] = —0.7561356880400178905356401098531772... 
—0.0737508631630889005240764944567675 ...7 
c[5, 2,15] = —0.2688595121851000270002877100466102... 
—0.84108577401329800103648634224905292...2 
c[5, —2, 15] = —0.2688595121851000270002877100466102... 

+0.84108577401329800103648634224905292...1 


I 


c[3, 1,24 


I 


I 


c[8, 1,9 


This yields: w7i(t) = t" — 2t6 -—#°+¢4+04+¢ —t-1. 
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Now we are able to attack our problem: 

We start with: A field Foam, a CM discriminant D for 2™, and the desired 
curve order N = 4n. 

We search: b € Foam such that the elliptic curve Y?+ XY = X°+4+56 
over Fam has order N = 4n.?! 


Recall n is a large prime number... 


1. Compute wp(t) and w(t) = wp(t) mod 2. 


2. Find the smallest divisor d of m greater than (log, D) — 2 
such that D is a CM discriminant for 27. 


3. Find p(t) = a degree d factor of w(t) (cf exercise (1)). 
4. Compute 2 = a root in Fam of p(t) = 0 (cf exercises (2), (3)). 


5. b= B if D=Omod3 
7" ~ ) 63 else 


This is a somehow enigmatic protocol. We shall try to give the necessary 
explanations in the section “Elliptic Curves with Complex Multiplication”. 


Exercises 


(1) Let the binary polynomial f(t) = fi(t)--- f-(t), r > 2, be a product of r 
irreducible factors of the same degree d. 


The following algorithm will produce a random degree-d factor of f(t): 

First the participants: 

g(t) =a divisor of f(t) whose degree will become — along several reduction 
rounds — smaller and smaller until we get the final random irreducible factor 
of f(t). 

h(t) = the end-of-round reduction of g(t). 

c(t) = a step-by-step approximation of h(t). 


The algorithm: 


1. g(t) = f(t). 
2. While degg(t) > d: 


1 An elliptic curve with the considered one-parameter equation — i.e. without the 
X?-term — necessarily has order N = 0 mod 4! 
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2.1 Choose u(t) = a random polynomial of degree 2d — 1. 
2.2 Set c(t) = u(t). 


2.3. The (trial of) reduction round: 
Compute d— 1 times c(t) = c(t)? + u(t) mod g(t) 


2.4 Compute h(t) = gcd(c(t), g(t)) 
h(t) is expected to be a shorter product of irreducible factors 
of f(t) than g(t). If this is not true or if h(t) = 1, return to 2.1. 


2.5 Else, choose between h(t) and g(t)/h(t): the shorter expression 
will become the new g(t). 
3. Output g(t). 


Now the exercise; show the following: 
Let u(t) be a multiple of one of the irreducible factors of g(t): 
u(t) = m(t)f;(t). Then f;(¢) will be a factor of h(t). 
Recall: a ) = gcd(a(t), b(t)) via iterated Euclidean division: 
a(t) = a(t), A(t) = b(t). 
5 While G(t) £0: p(t) = a(t) mod G(t) 
Reset a(t) = B(t), A(t) = ple). 


3. d(t) = a(t). 


(2) Finding a root in Fpm of an irreducible binary polynomial. 


If f(£) is an irreducible binary polynomial of degree m, then f(t) has m distinct 
roots in the field Fom. 


(a) Try an (almost word-by-word) copy of the foregoing algorithm in order to 
obtain a random root of f(t) in Fam. 
(b) Apply the algorithm to f(t) = ¢4+t3+¢?+t+1 and Fig = Fo[x]/(x++a+1). 


Help (a) First consider the adaptation of the previous algorithm to our 
needs: 


1. Set g(t) = f(t). 
2. While degg(t) > 1: 


2.1 Choose a random u € Foam. 
2.2 Set c(t) = ut. 


2.3. The (trial of) reduction round: 
Compute m — 1 times c(t) = (c(t)? + ut) mod g(t) 
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(b) 


(4) 
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2.4 Compute h(t) = gcd(c(t), g(t)) 
h(t) is expected to be of smaller degree than g(t). If this is not true 
or if h(t) is constant, return to 2.1. 
2.5 Else, choose between h(t) and g(t)/h(t): the shorter expression 
will become the new g(t). 
Output g(0). 


Put a= 2 mod (a4 +2+1). 

We know: f@) =@+P +7 +1415 (¢+ a )(¢+ at + 0%)(t +o"). 
Let us try to find one of these four roots by means of our algorithm. 
First, we choose u = a?. The reduction round yields c(t) = alt? + a°t? + 
t+a®. h(t) =a°t? +a07t+a"™. This is our new g(t). 

Now, we choose u = a? (one of the roots...) and get finally c(t) = 1. 

You are surprised? Note that we start with ut and not with t — u — so 
there is no reason for an analogy to the claim of exercise (1). 

Finally, we take u = a. Now, the reduction round yields c(t) = al8t+ a. 


h(t) = t + a® allows to conclude. 


Let f(t) = ¢” +---+ 1 be an (irreducible) binary polynomial (which 
admits m distinct roots in Fm). Let 61, b2 be two roots of f(t). 


BE, =Y7+XY = X° +b, 


[Se Geet oe @ Marae, Ga a 
Show that EF, ~ Fy (as abstract groups). 


[Hint First, observe that the set of roots of f(t) is of the form 
fb, b2,b+,...07" }. 

It suffices to show the claim for b and b?. 

Now show that the map EF, 3 (x,y) += (a?,y”) € Eg is well-defined, 


compatible with the group operations, and bijective. 

D = 23. 

(a) Compute wo3(t) = t? —t— 1. 

(b) D = 23 isa CM discriminant for 8 = 2°. We get over Fg = Fo[x]/(2?+ 
x +1): w3(t)mod2 = #+t+1 = (t+ a)(t+a7)(t+a4), with 
a=axmod (#?+2+1). 

This yields three elliptic curves over Fs: 


By eve xy = XA aa, 
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Ey =Y?7 + XY = X°+a® 


E3=Y? 4+ XY =X? +a’. 


Show that each curve is isomorphic to Z/12Z (as an abstract group), 
and find a base-point (a generator) in each case. 
(c) D = 23 is also a CM discriminant for 64 = 2°. 
Consider Fs as a subfield of Fg4 = Fa[x]/(a° + x+1) in the following way: 
Put @ = x mod (#6 +2+1) and a = 677 = 634 6?+. Then a?+a41 = 0. 


Thus Fg = F [a] C Fea. Furthermore, w3(t) mod 2 factors now as follows: 
P+t+1= (t+ 6?")(t+ B4)(t+ 6%) . We get over Fea: 


Bes Vr xy jx? eB, 


Bea V4 xY = Xia ge", 


E3 =Y74+ XY = X34 8°. 


The order of each of the three curves is 72 (why?). Find points of order 
24 on each curve. Are there points of order 72 (i.e. are the curves cyclic 
groups)? 


[ Solution: (a) Choose f(z) = 1—z—z?+25+z7—z!2—z!5 as an optimistic 
approximation of F(z). With 09 = exp(—V 23-7), 01 = exp(=Y23+i 7), 
62 = exp(—G21) 


we get 

fy (1,0, 23) ~ 05 74 oe c[1, 0,23] = Ze f ot 0, 23) 

£,(3,1,8) #0, %-£ and = ¢[3,1,8] = 4 -e2F - £,(3,1,8) 
a “a 197i 

£1(3 -1 8) & 05 H " £(62) c[3, —1, 8] = Fa O24 - £,(3, -1,8) 


(93) 
We approximately obtain: 
c[1, 0, 23] = 1.324717957, 
c[3, 1, 8] = —0.6623589785 + 0.56227951202, 
c[3, —1, 8] = —0.6623589785 — 0.5622795120%, 
(b) X? + 23Y? = 32 admits the primitive solution (x,y) = (3,1). This 
gives the order 9 + 3. Since we have chosen the option of an equation 
without X?-term, the order must be divisible by 4. Three base-points: 
1: G, = (a°,a), Eo : Go = (a?,a7), E3 : G3 = (a®,a3). (c) X27 + 
23Y? = 256 admits the primitive solution Bae (7,3). This gives the 
order 65 + 7. Now we can argue as before, or use (b): Since the group of 
Fg-rational points is a subgroup of the group of F¢4-rational points, the 
order of the latter must be divisible by 12. Three samples for points of 


154 2 Cryptography 


order 24 on Ey = Y2+XY = X34 638; P, = (G1, 931), P, = (B27, 68), 
P3 = (6°, 319). The underlying abstract group is not cyclic. 


(5) D=39. 
(a) Compute wag(t) = t* — 3t3 — 4t? — 2t — 1. 
(b) Show that D = 39 is a CM discriminant for 16 = 24 and for 256 = 2°. 
(c) First consider the scalar field Fig = Fo[z]/(a* + 2+ 1). Put a = 
x mod (24+2 +1). 
w39(t) mod 2 = t4+#2+1=(t+a7)(t+a1')(t+ a1%)(t+ a4). This 
yields four elliptic curves over F 16: 


BE, =Y?4+ XY = X$4a’, 
Bea VV? XY aX tag, 


hye VA XY S XP aes, 


E,=Y?+ XV = X34 a". 


Show that E, is a cyclic group of order 12, generated by G, = 
(a!?,a*). Find base-points (i.e. generators) for Ey, E3, E4. 

(d) Now extend the scalar field Fig to Fosg = Fo[2z]/(28 +24+23+27?+1). 
Put 8 =a mod (28 + xt + 23 + 2? +1). 
Fi6 = F[a] G Fase, with a = 17 = 10011000. 
Thus, over Fo56, we get 
w39(t) mod 2 = t4+ 43 +1 = (¢ + B19) (t + 187) (t + B72") (t + B98). 
Our four elliptic curves now read as follows: 


E, =Y74+ XY = X? + 10010011, 
Ey =Y74+ XY = X? + 11011100, 


E3 =Y7+ XY = X? + 01000101, 


E,=Y?74+ XY = X? + 00001011. 
Show that each curve has order 264. What about cyclicity? 


2 Recall the log table for F35¢ in the AES-section. 
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Help (a) We replace F(z) by f(z) =1—z-—2%7 +22 +27 -—z¥@— 2%. 
With 09 = exp(—V39-m), 01 = exp(—V39- 3), 02 = exp( 347), 


63 = exp( ==! n) 

we get 

fy(1, 0,39) © 05 24 34 Hoe c[1, 0, 39] = {45 . fo(1,0,39)} 

fo (3, 0, 13) ~ 6; 24 Len ng (eo tl= {-35 - £o(3, 0, 1s)" | 
£,(5, 1,8) © 052 Fa) c[5, 1,8] = {45 eo 45, 1,8)} 
f:(5, 1,8) ~ 0; : Fe) e[5, —1,8] = 4. eH 6,-1,8)}" 


You should obtain: 


, 1,8] = —0.1513878187 + 0.5290154688% 
c[5, —1, 8] = —0.1513878187 — 0.5290154688% 


(b) X? + 39Y? = 64 
admits the primitive solution (x,y) = (5, 1). 1024X?+822XY+165Y? 
reduces to X? + 39Y? ; the initial solution (a, y) = (1,0) finally be- 
comes (x,y) = (—7,5). (c) (d) follow the strategy of exercise (4). 
(6) D=95. 
(a) Find the eight reduced matrices [A, B,C] such that AC — B? = 95. 
(b) Compute wo5(t) = ¢8 — 2¢7 — 216 + ¢& + 2t* - #8 +¢-1. 


a 


2,11] = 0.8090169935 + 0.2096901122% 
t3, 13] = 0.2107051554 + 0.6954398639% 


| Answer (b) We get wos(t) = (¢ — -y - fo(1,0,95))(¢ + + - 
fo (5,0, INE Fe em -£1(3, 1,32))(f— Jy -£,(3, -1, 32 Mea 
we? - fo Se wy e@ ® + fo(9,-2,11))(t - Z-e? 
(8, 3, 13))(t— Jp - e7 =e £5 (8, =3;13)): 
The approximate values of the class invariants: 

c[1,0, 95] © 2.532681963 
c[5,0, 19] + —0.9146479740 
c[3, 1,32] © —0.8287391440 + 0.6954398630i 

(9 

[8 


a 
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Reduced class polynomials for small discriminants 


wp(t) 


103 
111 
119 
127 
143 
151 
159 
167 
183 
191 
199 
215 
223 
231 


239 


247 
255 


263 


271 
287 
295 
303 
311 


t—1 

pe 4 

et 1 

p-P-1 

t* — 32 — 4t? — 2t-1 
$e 98 9p OF Sa. 


t* — 2° 4 
t” — 21° 


el 
P+i+PA+eP7—-t-1 


t — 3t* 4 
t® — 138° 
t® — 2t7 


+23 — 2? +4-—1 
— 11t* +402 — 4¢? —t-1 
28 +25 4+ 244 #8 +t-1 


Pot 


3t? — 37 — 2t-1 


t® — 2147 — 26¢° 


14t° + 4t* — 114° — 67? —t-—1 


Te + 5t* — 402 + 207? -—t41 


10 — 4¢° 4.548 
t — 3t* — t? + 2t 
t'° — 6t® + 1248 


8t” + 9t° 
2p 
13¢” + 9£° 


3t° — 3¢4 + 6t? — 6t? + 3t—1 


3t® 


47t° 


P—-t—1 
196t” — 219¢° 


3t* 
146¢° 


? 


121¢° — 63t* — 7° —t? +t-1 


71t" 


941° 


6t? — 10¢* — 4¢° — 5t? —t-—1 


10¢? + 1147 —t—1 


ot® — 7¢7 — 1146 
157¢° — 98t* 4 


4t° 
53t° 4 


t? — 5t8 4 


8 — 6t!? + 10¢1+ — 16¢1° + 224° 


19¢8 + 11¢7 — 5t° — 4° + 5t* — 4t° + 2t-1 


+ 347 — 348 — 342 —t—1 


t4 — 6t!s 


+ 4¢1? + 1141? — 13419 — 74° + 16t8 — 4t7 — 13¢® + 8¢° + 344 — 6t? + 2t-—1 


e—5t7 +2 +t 
t?? — 138¢14 


33t'° + 517° 


Po 1 
724 + 34947 


4t? 


118¢° — 32¢° + 114¢* — 45¢° + 292? 


—4t+1 


—4t? + 4t-—1 
t® — 4t° — 7t4 


t — 6tt4 + 2413 + get? + at™ — 27229 + 1329 4 


7 


15t® — 4¢” — 20¢° + 13¢° + 5¢* — 403 


6t? — 3t—1 


t!? — 186¢14 


194¢1° + 83949 


702¢% — 287t" + 101246 — 912¢° + 51344 — 2214° 


+66t7 —11t+1 

8 — stl? + 16¢1! 
+6t—1 
t a} aa 510 = 


8t" + 94 —¢t 


6t® — 
t'4 — g¢!5 + o¢!? + 6tt! — 5tl° — 749 — 9¢8 + Ge" + 2t8 — t* — 094-37 +441 


— 27t'° + 38¢° — 36¢8 + 2247 — 124° + 13¢5 — 1944 + 214° — 15¢? 


5t® + 3¢7 + 6t8 + 30° — 3t4*# - BP -— PP -1 


5 _ 744 4. 10t8 — 717 + 3t-—1 


11° — 325¢° — 130 


2t® — 756t" — 720t° — 447¢° — 173t* — 46t? — 36¢? — 2t—1 


9 _ 4tt8 — 1617 
10t? — 13t® 


+ 


14¢° 


42t*> — 3g¢14 — att? + 10¢1? + 25¢4! + 184° + 92° + £8 
5t? — 27 —t—-1 


37t16 
8t4 
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Example For D = 942679 we have h = h(Q(./—D)) = 620, and 


157 


w(t) = wp(t) mod 2 = $820 fO16 $610 $806 $804 7902 4 4597 1 4596 


1593 4. 4592 4 4591 4 7589 4 4588 4 4585 1 4580, 4568 
4566 4 4565 4 4563 1 4560 4 4559 , 4558 1 4557 1 4556 
p555 4 4552 4 4550 4 4548 4 4547 4 4546 1 4545 | 4543 
po4l 4 7540 1 4539 4. 4536 1 4533, 4531, 4529, 4519 
p518 4 4512 4 4510 4 4507 4 4505 4 4503 1 4501 , 4498 
4496 4 4495 4 4492 1 7491 4 4488 1 4487 1 4486 | 4484 
1483 4. 4482 4 4481 4 4476 4 4475 4 4473 1 4ATL | 4467 
p166 4 4465, 4464 , 4462 1 4460 , 4459 , 4458, 4456 
p454 4 4453 4 4452 4 4451 4 4450 4 7448 1 4447 | 4443 
p442 4 4441 4 4440 1 4439 1 4438 4 4430 1 4429 | 4426 
p425 4 4424 1 4423 4 4422 1 4421, 4417, 4416 | 411 
p410 4 4409 4 4408 , 4407 4 4406 , 4403 1 4401 | 4398 
4395 4. 4392 4 4389 4 4388 4 4387 4 4383 1 4382 4 4379 
p377 4. 4376 4 4373 4 4372 4 4371 4 4368 1 4366 | 4365 
4360 4 4359 4 4358 1 4357 4 4355 4 4354 1 4353 4 4351 
p346 4 4345 4 4344 1 4341 4 4340 4 4335 1 4333 4 4329 
4328 4. 4327 4 4325 4 4323 4 4322 4 4320 1 4318 1 4317 
p314 4. 4313 4 4312 4 4311 4 4309 4 4306 1 4305 | 4304 
4302 4 4301 1 4300 1 4299 1 4297 , 4292 , 4290 , 4289 
4288 4. 4287 4 4284 4 4281 4 4276 4 4274 1 4273 4 4272 
4264 4 4263 4 4262 1 4260 1 4257 , 4255 1 4253 4 4252 
4250 4 4249 1 4248 4 4245 1 4244 1 4242 1 4241 | 4239 
4238 4 4237 4 4235 4 4234 1 4233 1 4228, 4225 , 4223 
p221 4 4219 4 4211 4 4210 1 4209, 4208 4 4207 4 4201 
4200 4 4195 4 4192 1 4191 4 4190 , 4189 1 4186 1 4185 
p180 4 4179 4 4176 4 4175 4 4172 4 4171 1 4167 1 4166 
piGl 4 4160, 4158 1 4157 4 4155 4 4154 1 4152 4 4150 
pi48 4 p47 4 146 4, 4145 4 4141 | 4140 4 4134 | 4133 
4130 4 4129 4 4128, 4126 4 4125 4 4123 , 4121 | 4119 
pil7 4 pll4 4 4110 1 4109 4 4108 , 4106 1 4103 , 4102 
g201 4 4100 4 499 4 496 4 493 4 491 4 487 4 483 4 482 
76 4 475 4 474 4g 472 4 47 4 470 4 769 4 68 1 467 

65 4 p61 4 p59 4 pd7 4 p56 4 p54 4 pdL 4 749 4 745 
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#3 #42 #41 $37 $36 $35 $33 $30 #27 


£24 4 422 4 420 4 f17 4 p16 4 413 4 412 1 410 1 46 


+741. 
This polynomial factors into four irreducibles over F2, each of degree 155. 


One of these is: 


gi55 4 4154 ae 7145 4 4143 4 7140 1 4139 | 4138 ne £137 4 4186 1 4134 | 4132 4) 4131 
$180 $129 £128 $127 $126 $124 $128 #121 $120 f118 fil? 116 


pil 4 plld 4 4104 | 4103 4101 4 497 4 493 4 491 4 489 4 488 4 486 4 485 4 482 
1 #80 4 475 4 474 4 472 4 470 4 466 4 764 1 462 4 461 4 60 4 455 4 54 1 451 4 450 


1749 4 431 4 429 4 427 4 426 4 423 4 422 4 419 1 418 4 416 4 415 | 414, 413, 41d 


$0 4 PP ttt. 


Call it p(t). If G is a root of p(t), then the curve 


Y?+ XY = X?+ 68 


over Fo15s has order 4n, where n is the prime 


n = 11417981541647679048466230373126290329356873447. 


Elliptic Curves with Complex Multiplication 


Up to now, we have tried to explain the essential algorithmic steps of the 
Lay—Zimmer method relative to a minimum of mathematical prerequisites of 
the reader. 

But there is some robust Number Theory interacting with Algebraic- 
Analytic Geometry which control our step-by-step arguments. 

So, let’s briefly sketch the underlying Mathematics. 


1. Complex multiplication. 

The pivotal structure is that of a lattice A = (w1,w2)z in the complex 
plane (note that w; and w2 have to be R-linearly independent). Two lattices 
A; and Ag are equivalent (homothetic), whenever Ay = aA, for some a € C*. 
Thus every lattice is — up to equivalence and reindexing of the Z-basis — of 
the form A = (1,7)z, with 7 € H. 

A lattice A (which is trivially stable under multiplication by integers) is 
said to admit complex multiplication whenever aA C A for some non-integer 
a €C. In this case, the admissible multiplicators a@ constitute an order R in 
some imaginary quadratic number field K = Q(/—D) 73, and A has to be 
(homothetic to) an invertible fractional R-ideal (inside K). 


3 i.e. R is a unitary subring of K, and a lattice — a free Z-module of rank 2. 
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Example Recall our summary on imaginary quadratic fields K = Q(/—D) - 
with D = 3 mod 4. 

We only considered fractional R-ideals for R = Rx = Zw], with w = 
14V=D ie. for the maximal order (the ring of integers) of K. 

A typical non-maximal order would be R = Z[/—D] (we dealt with it 
implicitely when considering quadratic forms with discriminant —4D). Note 
that non-maximal orders R will admit non-invertible fractional R-ideals. 

Every lattice A = (w1,w2)z C C gives rise to a function field: the field 
C(A) of elliptic functions for A: f(z) is elliptic for A <> f(z) is defined on 
C, except for isolated singularities, is meromorphic on C, and doubly periodic 
in the following sense: f(z+w 1) = f(z+we2) = f(z) (A is the period lattice 
for f(z)). 


The most important elliptic function for A is its Weierstrass o-function: 


Aad= std ( : 3 x)" A’ =A-{0}, z¢A 


The singularities of o(z) = (z; A) consist of double poles at the points of the 
lattice A. o(z) = @(z; A) satisfies the differential equation 
= go(A) = 60> 2, 2 
'(z)? = 4o(z)? — z) — 93 with 7 gal WEA! uA 
9"( ) 9( ) 92( ) 93 93 = 93(A) = 1400 cq oe 


A(A) = ge(A)3 — 2793(A)? is always non-zero. Thus j(A) = 1,728 - wat) 
is well defined. j(A) is an important lattice invariant: j(A) = j7(A’) => A 
and A’ are homothetic. 

Now consider the mapping C — A 3 z += (@(z), @'(z)) € C?. 

We get an (analytic) parametrization of the affine algebraic curve Y? = 
X3 — g9X — gz (an elliptic curve, given by its Weierstrass equation). 

Let E(C) be the projective curve defined by Y?Z = X? — goXZ? — 
g3Z°, together with its group structure given by the addition-of-points and 
doubling-of-points formulas (similar to those introduced at the beginning of 
this section). Then our (9, 9’)-parametrization induces a (biholomorphic) iso- 
morphism C/A ~ E(C). 


Caution We are dealing with two group objects in slightly different cat- 
egories: C/A is a complex torus, with its elliptic function field; E(C) is an 
abelian variety (a group object in the category of projective algebraic vari- 
eties), with its field of rational functions. But E(C) bears an analytic structure 
that makes it comparable to C/A. 


Uniformization Theorem: Let E = E(C) be an elliptic curve, given by the 
Weierstrass equation Y? = X?—go.X — 3, 92,93 € C, g3 — 2793 £0, 
then there is a unique lattice A Cc C such that go = go(A), g3 = g3(A). 

Thus we are able to speak of elliptic curves with complex multiplication, but 
have to refer to the lattice model. Now, let R be the ring of complex multipli- 
cators for A, then R identifies naturally with End(C/A) = the endomorphism 


160 2 Cryptography 


ring of the torus C/A. But this endomorphism ring has its counterpart in 
End(E(C)), the endomorphism ring of the abelian variety E = E(C) (note 
that a € End(E£(C)) => a is a rational mapping that is a group homomor- 
phism). This permits an intrinsic definition of an elliptic curve with complex 
multiplication: its endomorphism ring has to be non-trivial. 


Example The lattice A = (1,.\/—2)z admits complex multiplication by a = 
V2. 
The corresponding Weierstrass equation is FE = Y? = X° — 30X — 28. 


End(E) = R = Z[V—2]. 


In geometric language, the elements of R = End(E) are isogenies (i.e. 
surjective group homomorphisms with finite kernel). For a € R we have: 
deg(a) := #Kera = #(A/aA) = N(a) = the norm of a. In our case: 
deg(./—2) = 2. 

The complex multiplication by a = —2 reads in affine coordinates as 
follows: 


Qa7 +427 +9 1 Qu? + 8x —1 
V2 (ow) =( +) 


A(v+2) ° J-2 A(x+4+2)? 

(Note that x and y are coordinates relative to E — which explains the linearity 
in y.) 

2. Reduction of elliptic curves with given complex multiplication. A first ob- 
servation: Two elliptic curves Fy, and E> over C are isomorphic (as abelian 
varieties) if and only if the lattices A, and Ag of their torus models are equiv- 
alent (homothetic). 

Now return to our particular situation: Let D be a positive, square-free 
integer with D = 7 mod 8, R = (1, 14ivD), = the ring of integers in kK = 
Q(V—D). 

h = hp = the classnumber of K = Q(/—D). 

There are h non-isomorphic elliptic curves EF over C having complex mul- 
tiplication by R (i.e. such that R = End(£)): Namely, let 71,...,7, € H be 
the h canonical representatives for the ideal classes of K = Q(/—D), and let 
Ay = (1,71)z,..-An = (1, 71)z be the corresponding lattices. Then the associ- 
ated elliptic curves are non-isomorphic, and admit complex multiplication by 
R= (1, 4Y2)z. 

The j-invariants of the elliptic curves are given by j(7,) = j(Ak) 1 < 
k<ch. 

They are algebraic integers and generate the Hilbert class field Kp of K. 

Kp is the Galois extension of kK whose Galois group is precisely the class- 
group Cl(K) of K. The minimal polynomial of the j(7,),1 < k < h is the 


class polynomial 
h 


hp(t) = |] (¢- s(t) € ZEd. 


k=1 
We insist: hp(t) is a polynomial of degree h, with integer coefficients. 


Now consider |q = 2, and Fy = Fam. 
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If 4g = u? + Dv? (we know this theme...), then 2 splits in K = Q(V—D) 
(ie. 2R = P,P» is a product of two distinct prime ideals of R; and R/P, = 
R/P2 =F) and is unramified in Kp: 

There are r distinct prime ideals pj,... py in the ring Rp of integers of 
Kp such that 2Rp = p1---p, and Rp/pp=Fom 1<k<r. 


— Note that the field degree m must necessarily divide the classnumber h = 
hp. 


The reduction modp — where p can be any of the pz above — gives 


hp(t) = hi(t)-+- h(t), 


ie. we get a product of r irreducible binary polynomials h;(t) of the same 
degree m, each admitting m distinct roots in Fm. 

Important observation: The situation will remain unchanged if we replace 
hp(t) by some other minimal polynomial for any primitive element of Kp. 

Now accept the non-trivial fact that our complex elliptic curves with com- 
plex multiplication R C K = Q(./—D) admit defining equations with coeffi- 
cients in Rp C Kp. 

This makes them reducible mod p (where p is any of the prime ideals lying 
over 2). Thus we get h = hp elliptic curves defined over Fom. The fundamental 
observation comes from Deuring’s reduction theorem. Let Eg be any of these 
reductions, considered as an elliptic curve over Q = Foam = the algebraic 
closure of Fz. Then End(Eg) = R. 

(Reduction is conservative with respect to complex multiplication! 

The h = hp elliptic curves over Fgm, obtained by reduction mod p, will be 
precisely (up to isomorphism) the curves over Fm with complex multiplication 
by R. 

This leads us to adopt the following strategy: 

Compute the class polynomial hp(t) € Z[t] and reduce it modulo 2. 

It splits completely over Fm, and each of its roots is (more or less) the j- 
invariant of a looked for elliptic curve ( we never shall rigorously define the 
j-invariant for our characteristic 2 constellation; only note that whenever 
we deal with an equation of the form Y? + XY = X° +5, b # 0, then 
we have b= j~'). 

Unfortunately, hp(t) has very large coefficients. In order to remedy to this 
situation, it is possible to use a different polynomial wp(t) € Z[t] ,which also 
defines the Hilbert class field Kp. We only have to replace the function j by 
another well-chosen modular function.?° 


ees 


24 Do not be afraid of complex numbers acting on curves defined over Fam ! Why 
should the mapping P +— —7P not be the square of a “simpler” mapping P +— 
J/-7:-P? 

25 f(r) is modular whenever it is meromorphic on H and invariant under the action 
of SL2(Z). 
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Recall wp(t) = ieee: —c(T,)) € Zt], where the class invariants c(tT,), 
1 <k<h, are defined by means of the classical Weber functions. We would 
not get lost in technical details; let us altogether indicate the ingredients: 
Start with Dedekind’s 7-function: 

For rt € H, we put gq = e?'7. Let G, =e 
defined by 


Qri 


m . The Dedekind 7-function is 


n(7) is defined (converges) for 7 € H. The classical Weber functions f, f,, 
and f,; now are defined in terms of 7 as follows: 
f(r) = Ge wre 


‘ nr)? 
fi(r) = MEP 
27 
fo(7) = V2 . a 


Now, every modular function is a rational function of j(7) (considered as a 


3 
function on the upper half-plane). The identity 7 = (=) shows that the 


Weber functions are sufficiently general to support all modular arithmetic. 
For example, the reader will note their massive intervention in the defini- 
tion of our class invariants — up to normalizations which partially stem from 
our 4D-approach to the classgroup representatives. 
What is the advantage to replace hp(t) by wp(t)? 


Example D= 71. 
hri(t) =t" +5-7-31-127- 233 -9769t° — 2-5-7 - 44171287694351t° 
+2-3-7-2342715209763043144031t4 
—3-7- 31 - 1265029590537220860166039t* 
+2-7-11° - 67 - 229 - 17974026192471785192633t7 
—7- 11° - 17° - 1420913330979618293t 
+(11° - 17? - 23. 41-47-53)? 


(all primes <1,000 are factored out of the coefficients). 
w(t) =¢t — 2-2 +¢4+84+¢—-t-1. 


Our final point is to briefly explain the background for the system of the 
two equations establishing D as a CM discriminant for 2”. Consider an elliptic 
curve E = E(C) over the complex numbers, with complex multiplication by 
Rc K =Q(V—D). Suppose that E = E(C) is reducible to E = E(Fam). Let 
Q = Fp» = the algebraic closure of Fam. 

By the Deuring reduction theorem, R = End(£) = Ende(E a). (Reduction 
does not affect complex multiplication.) Now consider 7, € Ende(£a), given 


by T(x, y) — (x7, y"), q =F oi, 
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This is an isogeny of degree?® gq = 2”, and its fixed point set is precisely 
E(Fom) (the points with coordinates in Fy»). Thus: Ker(1 — 74) = E(Fom) 
and deg(1— 14) = N = #E(Fom). 

When “lifting” a, to 7 € R, we dispose of a number theoretic inter- 
pretation for the degree of an isogeny (which is a complex mutiplicator): 
deg(7) = N() = the norm of a. 

Since reduction preserves the degree (of separable isogenies), we arrive at 
two norm equations 

N(m) =q=2™, 

N(l-a)=N=#E(Fom). 

Written down more explicitely, they become our familiar system for CM 
discriminants. 


The Algorithm ECDSA 


We shall present the algorithm ECDSA without going too much into detail. 
Nevertheless, we shall insist on its structure as an additive twin of DSA. 

First the elliptic data, which will be public: 

Let us start with a field Fam = F[2]/(p(x)), where p(x) is an irreducible 
trinomial or pentanomial — so, the arithmetic of the field of constants will be 
public... 

Choose an elliptic curve defined over Fom (note that there are approxi- 
mately 2” mutually non-isomorphic elliptic curves over Fym — we can safely 
choose...): E(Fam) : Y7+XY = X°+aX?+b with a,b € Fom, b £0. Then, we 
need a base point, generator of a |(cyclic) subgroup of prime order] in €(F gm): 

G = (Xe, Ya) €E(Fom) such that nG = O, where n is a (large) prime 
number: n > max{2!©, 4,/2™}, 

Note that it is the prime number n which is the main security parameter 
for ECDSA?’. 


— Generation of the keys: 

d= The private key of the signatory: a “sufficiently arbitrary” integer 
between 1 and n— 1. 

The pseudo-random generation of d is guaranteed by auxiliary algorithms 
in the spirit of those that we have seen serving the DSA. 


26 The degree of an isogeny is the cardinality of its kernel. 

27 Tn order to guard our curve against existing attacks on ECDLP — the Elliptic 
Curve Discrete Logarithm Problem (to be explained in a moment) — the condition 
on the size of the prime n has to be complemented by two further conditions — the 
Menezes—Okamoto-—Vanstone Condition and the Anomalous Condition — which 
will guarantee that no transfer to an easy Discrete Logarithm Problem in some 
scalar field Fy.1 is possible. 
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Q = the public key of the signatory}: a distinguished point on the elliptic 
curve; 
more precisely: @Q = (XQ, Yq) =dG. 


— Generation of the signature o = (r,s). 
Let M be the message (the document) to be signed. 
(a) Computation of e = SHA-1(M). 
(b) Selection of k = the secret key of the signature act. k will be an integer 
between 1 and n — 1, “sufficiently arbitrary”, exactly like d. 
(c) (X1,¥Y1) = kG. 
(d) X1 is considered (by means of its binary notation) as an integer. 
Put then |r = X, mod n. If r = 0, return to b. 
s=k-'(e+dr)modn.|_ If s =0, return to b. 
(e) o = (r,s) =the signature (a couple of two integers). 


— Signature verification. 


The scenario is precisely the same as for DSA: The verifier of the signa- 
ture receives the transmitted message M’ along with the transmitted 
signature o’ = (r’, s’). He will carry out essentially the same operations 
as in DSA. 
(a) Computation of e’ = SHA-1(M’). 
(b) Verification that 1 < r’,s’ <n—-1. 
(c) Computation of w = (s’)~+ mod n. 
(d) Computation of 
u, = e’w mod n 
ug = r’w mod n 
(e) Computation of (X71, Y1) = uiG@t ueQ. 
(If urG = —u2Q, he rejects...) 
(f) If r’ = X,1 (as integers), he accepts; otherwise, he rejects. 


Summary of the Analogies Between DSA and ECDSA 


1. The relevant group information 


Group (Zip) E(Fen) 

Group elements |The integers {1,2,...,p —1}|/The points (X,Y) 
which satisfy the 
equation of the elliptic 
curve, with the point 
at infinity O 

Group operation |Multiplication modulo p Addition of points 


Notation The elements: g,h The elements: P, Q 
The multiplication: g-h The addition: P+ Q 
The exponentiation: g® Scalar multiple of 
a point: aP 
Discrete logarithm|Given g € (Z/pZ)* and Given P € E(Fom) and 
problem h = g* mod p, find the Q = aP, find the 


integer a. integer a. 


2. The basic notation 
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DSA Notation|ECDSA Notation 
q n 
g G 
x d 
y Q 


3. The algorithmic skeleton 


DSA Setup 


ECDSA Setup 


q divides p — 1. 


2. g is an element of order q 
in (Z/pZ)*. 


3. We work in the group 
COG ecdege 


1. p and q are prime numbers,|1. E is an elliptic curve, defined 


over the field Fam. 


2. G is a point of prime order n 
in E(Fom). 


3. We work in the group 
{O, G,2G,...,(n — 1)G}. 


4. The key generation 


DSA Key Generation 


ECDSA Key Generation 


x between 1 and q—1. 


3. The private key is x. 


4. The public key is y. 


1. Select a random integer |1. Select a unpredictable integer 


2. Compute y = g” mod p.|2. Compute Q = dG. 
3. The private key is d. 


4. The public key is Q. 


d between 1 and n — 1. 


16 
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5. The signature generation 


DSA Signature Generation ECDSA Signature Generation 

1. Select a random integer 1. Select a unpredictable integer 
k between 1 and q—1. k between 1 and n — 1. 

2. Compute g* mod p. 2. Compute kG = (X14, Yi). 


3. Compute r = (g* mod p) mod g. |3. Compute r = X1 mod n. 
4. Compute m = SHA-1(M). 4. Compute e = SHA-1(M). 
5. Compute s = k~'(m+ ar) mod q.|5. Compute s = k~'(e + dr) mod n. 


6. The signature for M is (r,s). 6. The signature for M is (r,s). 


6. The signature verification 


DSA Verification ECDSA Verification 


1. Compute m = SHA-1(M).  |1. Compute e = SHA-1(M). 
2. Compute s~! mod gq. 2. Compute s~! mod n. 

3. Compute u,; = ms! mod g. |3. Compute u; = es! mod n. 
4. Compute uz =rs~'modq. |4. Compute uz = rs~! mod n. 
5. Compute v’ = g“y“2 mod p. |5. Compute uiG + u2Q = (X1,V). 


6. Compute v = v’ mod q. 6. Compute v = X; mod n. 


7. Accept the signature if v= r.|7. Accept the signature if v =r. 


Exercises 


(1) Assume a correct transmission of the message M and of the signature 
o = (r,s). Adopt the standardized notation above. Show that you will get 
necessarily: 

r = X, mod n, where X, is the abscissa of the point uiG + u2Q on the 
given elliptic curve. 
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(2) Let Fy¢ = Fo[x]/(a* + x +1), and let w = 2 mod (x* + x +1). Consider 

the elliptic curve €(Fig) : Y2 ++ XY = X3 +0. 

(a) Let G = (w,0). Verify that 5G = O. 

(b) The message digest of the received message M’ is e’ = H(M’) = 1100 
(four bits — we shall be modest...). The transmitted signature o’ = 
(r’, 8’) = (2,2). 
The public key of the signatory: Q = (w°, w°). 
Show that Q € E(Fi¢), and verify the signature. 


The Security of ECDSA 


The security of the system ECDSA depends on the algorithmic complexity of 
the elliptic curve discrete logarithm problem (ECDLP): 


Given G,Q € E(Fam), nG = O, find 


O0<l<n-1  suchthat Q=IG (if ] exists) 


The best general algorithms known to date for ECDLP are the POLLARD- 
rho method and the POLLARD-lambda method, which take about ,/7n/2 
and 3.28,/n steps (one step means one elliptic curve addition), i.e. — according 
to the requirements of the standard concerning the size of n — at least 2°! steps 
(recall that we have to exclude certain “vulnerable” elliptic curves...). 

Which is the power of the actual technology? Concerning software attacks, 
assume that a 1 million instructions per second (MIPS) machine can perform 
4 x 10? elliptic curve additions per second. Then, the number of elliptic curve 
additions that can be performed by a 1 MIPS machine in one year is 2°. 
As a consequence, if m = 163 (that is the degree of the scalar field) and if 
n ~ 2/60 then 10,000 computers, each rated at 1,000 MIPS, will find (with 
POLLARD-rho) an elliptic logarithm in 85000 years. Concerning hardware 
attacks, the reference is a study of van Oorschot and Wiener (1994) which 
estimates that if one uses a special-purpose hardware for parallel search, and 
that if n ~ 10°° ~ 2!°°, then a machine with 32,5000 processors (that could 
be built for about 10 million dollars) would compute a single discrete elliptic 
logarithm in about 35 days. This is the reason for the standard to require that 
n > 2169: so, hardware attacks of this kind would be infeasible. 


An Example of ECDSA 


Our scalar field will be F210. = F2[x]/(at9! + x? + 1). The elements of this 
field — which are binary words of length 191, i.e. the coefficient-sequences of 
the remainders of division by p(x) = x!°'+2°+1 — will be presented in blocks 
of six 32-bit words, in hexadecimal notation. 
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2 Cryptography 
The elliptic curve: 


E(Foi1) : Y27 + XY = X34+aX745b 
with 
a = 2866537b 67675263 6a68f565 54e12640 276b649e £7526267 
b = 2e45ef57 1f00786f 67b0081b 9495a3d9 5462f5de Naal85ec 


The base point G = (XG, Ya): 

Xo = 36b3daf8 a23206f9 c4f299d7 b21a9c36 9137f2c8 4aelaad, 

Yq = 765be734 33b3f95e 332932e7 Oea245ca 2418ea0e £98018fb, 

nG = O, 

with 

n =1569275433846670190958947355803350458831 20559545 1630533029. 
#E(Fo101) = 2n. 


Key Generation 


d = 1275552191113212300012030439187146164646146646466749494799. 
Q =dG= (XQ, Ye) with 

Xg = 5de37e756bd55d72e3768cb396 f feb962614dea4dce28a2e7, 

Yq = 55c0e0e02 f5 fb132ca f416e f856229bbb8e1352003125bal1. 


Signature Generation 


=, bt 


The message M = “abc” (familiar to us...) 


1. 


2 


4. 


The message digest: 
e = SHA-1(M) = 968236873715988614170569073515315707566766479517. 
Elliptic computations: 
2.1 Selection of k: 

k = 1542725565216523985789236956265265265235675811949404040041. 
2.2 Computation of R = kG = (Xi, Yi): 

X1 = 488e5a11 fb55e4c65471dcd49e266142a3bdf 2b f9d5772d5, 

Y, = 2ad603a05bd1d177649 f9167e6 f475b7e2 f f590c85a f15da, 
2.3 X 1 becomes an integer: 

X1 = 1656469817011541734314669640730254878828443186986697061077. 
2.4 r= X, mod n: 

r = 87194383164871543355722284926904419997237591535066528048. 


. Congruence computations: 


Computation of s = k~!(e + dr) mod n: 
s = 308992691965804947361541664549085895292153777025772063598. 
The signature o = (r,s): 


r = 87194383164871543355722284926904419997237591535066528048, 
8 = 308992691965804947361541664549085895292153777025772063598. 
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Signature Verification 


We suppose a correct transmission. 


1. 


2. 


The message digest of M’: 
e’ = SHA-1(M") = 9682368737159886 141705690735 15315707566766479517. 
Elliptic computations: 
2.1 First w = (s’)~+ mod n: 
w = 952933666850866331568782284754801289889992082635386177703. 
2.2 Now uw, = e’w mod n and ug = r’w mod n: 
uy = 12488864071547078540224345 16084062503301792374360994400066, 
ug = 527017380977534012168222466016199849611971141652753464154. 
2.3 Then (X1,¥Y1) = u,G + u2Q: 
(u1G) x =1a045b0c 26af1735 9163e9b2 bflaa57c 5475c320 78abel59 
(u1G)y = 53eca58f ae7a4958 783e8173 cflcal73 eac47049 dca02345 
(u2Q) x =015cf19f e8485bed 8520ca06 bd7fa967 a2ce0b30 4ffcfOf5 
(u2Q)y = 314770fa 4484962a ec673905 4a6652be 07607d93 cac79921 
X= 438e5all fb55e4c6 5471dcd4 9e266142 a3bdf2bf 9d5772d5 
Y, = 2ad603a0 5bd1d177 6499167 e6f475b7 e2ff590c 85afl5da 


. Validation of the signature: 


3.1 X 1 becomes an integer: 

X1 = 1656469817011541734314669640730254878828443186986697061077. 
3.2 v= X; mod n: 

v = 87194383164871543355722284926904419997237591535066528048. 
3.3 v=’. ok. 


3 


Information Theory and Signal Theory: 
Sampling and Reconstruction 


In this chapter, we shall treat the following question: what is the digital 
skeleton of a (traditionally acoustic) signal, which describes it faithfully and 
exhaustively? In a more technical language: we shall deal with conversion 
(without loss of information) 


analog —— digital —> analog 


in signal theory. 

This means appropriate sampling, followed by reconstruction of the ini- 
tial signal from the time series of its samples. The Sampling Theorems are 
always interpolation theorems (in a rather general sense: there will also be 
convergence questions — with sometimes delicate nuances. ..). 

In principle, we could also adopt the following viewpoint: the reality is 
discrete (perhaps even finite); the “continuous world”! is the bridge between 
perception and modelling (simple forms described by simple laws). In this 
sense, a Sampling Theorem will control the uniqueness of the law (of the 
form) behind the empirical data — permitting to refine arbitrarily their “digital 
density”. 

But there are also classical down-to-earth arguments around digitization, 
i.e. conversion of a continuous-time waveform into a bitstream. First, a sampler 
will convert the continuous-time waveform into a discrete-time sequence of 
numbers (a time series). This process is a mathematical abstraction, and we 
will stop precisely here. Then, the resulting time series has to be quantized 
into a bit stream (how many bits per sample?) — a non-mathematical step 
that we shall neglect. At any rate, digitizing means loss of information (at 
least for those who believe in a perfect control of the continuous situation). 
But, miraculously, it is exactly at this point that the discrete world wins a 
strange battle against the analog world: analog channels have a finite capacity 
and often inflict considerable distortion due to noise; properly digitizing the 
source and transmitting a digital signal through the channel often results in 


' The continuous event “film” is nothing but a time series of digital images. 
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less distortion than if the source output were left in analog form. So, perversely, 
digitization can even be considered as an information preserving process. 

We shall proceed in the following way: 

At the beginning, we shall introduce one of the main tools for subsequent 
arguments: the Discrete Fourier Transform. Then, we shall improvise on the 
Sampling Theorem in the periodic case. Finally, we shall attack the non- 
periodic case, where the Sampling Theorem (due to Whittaker, who proved 
it in 1935) has always been associated with the names of Shannon and of 
Nyquist. 

Let us repeat: we shall always remain in an ideal mathematical world: 
quantization as well as distortion will be excluded. 


3.1 The Discrete Fourier Transform 


This fundamental section treats a chapter of Linear Algebra which is more or 
less the mathematical theory of periodic time series (i.e. sampled trigonometric 
polynomials). 

So, everything is dominated by the reassuring image of the unit circle 
and its polygonal approximations. The great coherence and elegance of this 
theory go hand in hand with a remarkable robustness towards a lot of inter- 
esting applications of non-periodic character, in Discrete Mathematics and in 
Engineering. 


3.1.1 Basic Properties 
We shall choose an ad hoc approach and treat the Discrete Fourier Transform 
as an object of lectures in elementary Linear Algebra. 


(a) Prelude: The nth Roots of Unity 


Consider S! = {z € C: |z| = 1}, the unit circle in the complex plane. There 
is a “natural” 27-periodic parametrization given by 
el? —cosO+i-sind, OER. 


Fundamental Identity 


ei(A1+62) — gigi 9, 0) ER. 
Let us verify: 


ell@i +02) — cos(6; + 62) +i-sin(61 + 62) 
= cos 6, - cos 62 — sin 6; - sin 62 + i(sin 61 - cos Ag + cos 6; - sin 2) 


= (cos 6, +i- sin 01)(cos 62 +i- sin 02) = ele'%. 


A well-known consequence of our fundamental identity is the 
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De Moivre Formula 
(cos 6 +i-sin6)* = cos(k0) +i-sin(kO), ic. (e!)* = e'(*®) for every k € Z. 
The argument: 


For k > 1 one obtains the De Moivre Formula by recursion on k (since 
k0) — @i@ . ei(k—1)6) 


. As to negative exponents, observe only that + ae 


(inverse = conjugate) for z € St. Let us insist: ei? =e? = +. 


ell 


ss _ 2mi ee 
Definition For n> 1, we put w, =e ™ =cos 2m —i-sin on 
This gives: 
we = —l, W4 = —-1, 
Li 1 
w3=—-5—4V3, we = §V2- iv?2. 


Consider now, for fixed n > 2, the n (distinct) complex numbers 
Wn, W2,ws,...,w™ = 1 on the unit circle. Geometrically, we get (for n > 3) 
the n vertices of a regular n-polygon inscribed in the unit circle. Algebraically, 
we obtain the n distinct roots of the polynomial X” — 1. 

This is the reason for their name: the nth roots of unity. Note that with 
increasing powers of w,, we turn around the unit circle clockwise (this is a 
consequence of the “conjugate” definition of w,,). 


Examples 


(1) X4-1 = (X—a4)(X —w?) (X —w3)(X —wt) = (X+i)(X4+1)(X-i)(X—-1). 
(2) The eight 8th roots of unity: 


B= wy 8 =—2V2— V2, we =wt =I, 
w§ = —pV24 5V2, of swf =i, wf =5V2+3Vv2, wh =1. 


We note that 


Back to the general case: we have, forn > 2: w* = we ek=k' modns 
k — k’ is divisible by n. 


Example wz! = Ga = We = —We. 
Exercises 
Compute (the real number) a = (2 — 2i) - w§°. 


) 
) Compute z = wi! -itwg,?. 

) Compute the sum 1 + w3 + wf + w2 + w}? 
) 


4) Find the k € Z such that w} - wf = wag. 


1 
2 
3 


( 
( 
( 
( 
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The Discrete Fourier Transform of order n 


Fix n > 1, and consider the C-linear transformation 


Fy: C" — C", 
Z0 Zo 
zy Zy 
7 
zn—-1 Zn-1 


given by Z; = Rae wikz,, 0O<j<n-1. 
In matrix notation: 


he 1 1 1 1 20 
Z lw we wrk 
1 n n n ZY 
Zo 7 w wit weir) Zz 
Zn-1 1 wrt er os oe ie) en-1 


We shall design by F;, as well the particular matrix as the linear transforma- 
tion defined by it. 


Hence, F, = Cale cme 
Note that F, : C — C is the identity. 


Example 

Ted, dee el 
1-i-1 i 
1-1 1-1 
1 i -1 -i 


Fy = 


The equations for the Discrete Fourier Transform of order 4 are then: 


Zo = % + 21 + 22 + 23, 


Z\ = 20 — iz, — 29+ 123, 


22 = % — 2+ 22 — 23, 


23 = 20 + iz — 22 — iz3. 


Exercise 


Write the matrix Fg in function of +1, +i, twg, +Ws. 
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One obtains 


11 1 1 721 727 «21 ~«21 

1 Ws -i —We —1 —Wes i Weg 
1 -i -1 i 1 -i -1 i 

1 —We i Ws —1 We -i —Ws 
1-1 1 1 il 1 il 1 
1 —We —i Weg —1 Ws i —We 
1 oi -1l -1 1 i -l -i 
1 Weg i —We —1 —Weg -i Ws 


Let us return to the general situation. Our first goal is to show the following 
result: For every n > 1, Fy, is an invertible linear transformation, and the 
inverse transformation F'~! is “very similar to F,,”. 


Fundamental Observation 


Let z€C,z#land z”=1. Thenl+z4+274+---+2%1=0. 
The argument is simple: 1 — z” = (1— z)(l+z+27+---+2"74). 


Now we are able to be specific. 


Proposition n> 1. Then F,, is invertible, and F~+ = +F,, (componentwise 


conjugation), 1.e. 


i 
n 


1 1 1 1 
wt wr? a 
1 9 _4 —2(n—1) 1 : 
Rots - 1 ve on = — (wy? )o<j,k<n-1- 
[ie (n-1) on 2(n-1) ie (n—1)(n—1) 


Proof We have to show that +F,, -F, =I, =the identity matrix of order n. 
Let 1; be the jth row of F,, and let cz, be the kth column of F,,0< ZE< 
n—1. 


1 for j =k 
- a ee 
Let us show that 41, + cx ie forh #k. 
But 
1 
wr 
ok 
“1; -ck==(1u/ wid. wht09 ) se 
n n 
Pac 
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n 


For j = k we obtain * = 1, as needed. For j # k we get the sum +(1 t+2+ 
ete. 42771) with z= wi-* A 1,2" = 1. Thus our foregoing observation 
permits to conclude. 


Example 
1111 
1/1 i —-1 -i 
es ems 
ie 1-1 1-1 ]’ 
1-i-1 i 


i.e. we have the backward equations 


20 = +(Zo oe Z\ ae Lo + Z3), 
21 = (Zo +iZ — Z2 —iZ3), 
za = 4(Zo — 21 + Z2 — Zs), 
23 = +(Zo iZ, Z2 t iZ3). 


Note that the particular form of F,+, very similar to F;,, allows the “algo- 
rithmic elimination” of F771. (The algorithms which compute y = F,,(a) can 
be used for the computation of « = F7'(y)). 

More precisely, we have the following result: 


Fi1(y) = +F.@), 


n 


i.e. let 
20 Yo Yo 20 
2 Yi Y1 1 ZL 
SHy|, then Fy =— 
n 
zn—-1 Yn-1 Yn-1 en—1 


We insist: The inverse Discrete Fourier Transform is — up to two compo- 
nentwise conjugations — reducible to the Discrete Fourier Transform. 
The argument is simple: 
In generalizing the identity @Z = G-Z (for a, z € C) we obtain, for A € C"*” 
and 
20 
— € C”, the following identity: Az = AZ 


en—-1 
(the conjugation of a vector (of a matrix) is done componentwise). 


Let us show our claim for n = 2: 


Goo G01 Zo \_ f @o0o2%0 + Go1%1 \ __ f @0-2% +401° 271 \ _ f a0 Goi 20 
a10 Git 21 a1020 + 41171 @i0 + 20 + G11: 271 a10 Q11 Zz 


In our particular situation, we get 


Fly) = +F,(y) = +F,(9), as promised. 


n ” a 
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Exercises 


i) 
NaN 


Nai Sur 


Write the matrix Fy ' in function of £1, +i, +wg, +s. 

Compute the matrix Jy = 5 F?. 

Find the matrix J, = iF (follow the proof of the foregoing proposition). 
Answer: 


100--000 

000--001 

a Paneeeane E 

001--000 

010--000 
ZO ZO 
eo ea el ne =| fet 
en-1 21 


Show that Fy! = 4F°. 

n > 2. Let 1; be the jth row of the matrix F,,0<j <n-1. 
Verify, for the matrix Fx, that l7 1,, lg ly, 1; 1s, ly ly. 
Generalize: show that 1,_; = 1 forl<j<n-1. 


The Convolution Theorem 


The Convolution Theorem describes the characteristic property of the Dis- 
crete Fourier Transform. Stated differently: the very definition of the Fourier 
transform encapsulates this result. 


In the beginning, we shall adopt a purely formal viewpoint. We shall re- 


main in the context of Linear Algebra, without reference to the language and 
the arguments of signal theory and of digital filtering. 


Recall The circular convolution. 


B20) 
Ty 
Forz=| 22 € C”, the associated circular matrix C(x) is defined by 
Tn-1 
To Ly-1 Ln—-2 Ty 
Ty TO Mn-1 r2 
C(x) = XQ XY XO X3 


Tn—-1 Un—2 Un—-3 °°" VO 
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The multiplication of a circular matrix with a vector presents important par- 
ticularities: 


(1) Commutativity. C(a)-y = C(y)- 2. 
(2) Invariance under cyclic permutation. C(x) - a(y) = o(C(x) - y), where 


20 2n—1 
21 20 
(on = : 
zn—-1 zn—2 


Combining (1) and (2), one obtains 
(3) If z = C(x) -y, then C(z) = C(x)C(y) = C(y)C(x) (multiplication of 
circular matrices). 


Definition The (circular) convolution product *:C"™ x C? —> C”. 
Let us define, for 


nat) Yo 
x 
a 1 y= Y1 € cn 
Tn-1 Yn-1 


their (circular) convolution product z= xx*y by z= C(«)-y=C(y)-a. 
We immediately get: 


(1) cxy=y*e, 
(2) o(a*y) = (o(x)) *y = x* (o(y)), 
(3) C(x *y) = C(x)C(y). 


Note that all this is already well known: recall the operations on the words 
(the columns of the state array) in the cipher AES—Rijndael. 


Observation Polynomial multiplication via circular convolution. 
P(T) = ao + T+ +++ +G,1T™™, 
q(T) =bo +0,T+---+ by_1T”!. 


Put 
ao bo 
ay by 
A= An-1 ’ B= bn—1 € cr, 
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Compute 


C=AxB= rea Ogun 
C2n—2 
0 
Then: p(T)q(T) = co te1T + +++ + Con—2T?"?. 


In this way, we obtain the coefficients of the product polynomial by means 
of circular convolution. The practical interest of this “formal pirouette” comes 
from the fast algorithms for the computation of circular convolutions. This will 
be our next main theme. 


Exercises 
3 -—1 
—2 2 
(1) Compute 0 * 5 
4 1 
2 1 
3 0 
—2 —2 
(2) Compute : * : 
5 0 
—7 —2 
4 0 
(3) Solve the equation a* x = b 
0 XO —2 
é = 1 a Ly a —2 
with a= 9 | t= oe b= 9 
3 x3 2) 
Notation Let fo,f,,...,f,-1 be the n columns of the matrix F,,. (F, is 


invertible; thus the n columns of F;, are a basis of the vector space C”). 


(4) Let 
0321 
1032 
ae 2103 


3210 


Show that {fo, fi, fo, f3} is a basis of eigenvectors for A, and compute the 
corresponding eigenvalues. 

(Recall: v is an eigenvector for a matrix A whenever v is non-zero and 
Av = Av, where the scalar \ is the associated eigenvalue: “A acts by 
multiplication with the scalar » in direction v”). 
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(5) Let 
00...01 
10...00 
a= |]01...00] ec”. 
00...10 
Show that {fo,fi,...,f,-1} is a basis of eigenvectors for o in C”, and find 
the corresponding eigenvalues. 
ao 
(6) Let A=C " be a circular matrix. 
an-1 


(a) Show that A = aol + ayo + ago? +++» +4@n_10"1. 
(b) Show that {fo,f1,...,f£n-1} is a basis of eigenvectors for A in C”, and 
find the corresponding eigenvalues. 


Now we are ready to present the result which describes the characteristic 
property of the Discrete Fourier Transform. 


Let us first fix the notation: 


az) Yo 20 
X11 Y1 21 
L= , y= . >, 2ecey= EC, 
Tn-1 Yn-1 zn—-1 
Xo Xo Yo Yo 
XxX r4 Yi Y1 
X= oa fF, , Y= ‘ = Fy ) 
Nps En-1 Yn=1 Yn-1 
Zo Z0 
Ly 21 
Z= : = Ff, 
Zn-1 zn-1 


Then, we are able to state and prove the Convolution Theorem 


1.e. 
Zo XoYo 
Zy X1Y1 


Zn-1 Xn—1Yn-1 
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Proof Convention: all indices have to be interpreted modulo n: we shall replace 


a negative index by its (non-negative) remainder of division by n. 
We have, for0<j<n-1: 


jk ik 
Z,= y Zw” = y (> x ~ ) wi 
7 re re aca 0<k<n-1 0<m<n-1 mUk—m } Wn 
es S- ie 
=> x —mW “é 
Sey m( o<k<n—1 4 m 


Let us make now vary p = k — m between 0 and n; then we get 
45 = ose Lmwi™ - ee YpwiP = X,Y;, as claimed. 


If we write now — not without ulterior motives — our theorem differently, 
then it may be read like this. 


Convolution Theorem bis: The (circular) convolution product of two 
vectors x, y € C” can be computed in the following way: 


Remark The computation of a circular convolution according to the initial 
definition needs n? complex multiplications (where n is the length of the 
vectors to be multiplied). Using the foregoing identity, a zero-cost Fourier 
transform would reduce all this to n parallel complex multiplications. On the 
other hand, by traditional matrix algebra, we shall need a priori 3n? +n 
complex multiplications. 

But admit for the moment the following result. 

If n = 2™ — n is a power of 2 — then there exists an algorithm (the Fast 
Fourier Transform) which reduces the computation of y = F,,(x) tom-2™~1 = 
sn-Logyn complex multiplications. The computation of a convolution product 
of two vectors of length n = 2™ will then need only (3m + 2) - 2"~! complex 
multiplications (recall the “algorithmic elimination” of F). For example, 
for n = 2° = 256, the direct method needs 65,536 complex multiplications, 
whereas the fast method will need only 3,328 complex multiplications. 

There remains nevertheless the question: why are we interested in com- 
puting convolution products? The “natural” answer is given by signal theory 
(digital filters). We shall present it in a moment. But we dispose already of 
a rather convincing example, which comes from an absolutely non-periodic 
world. 


Fast Multiplication of Large Numbers 


(You may think of digital signature arithmetic, but the magnitudes are not 
really big there.) 

The multiplication of two (large) integers p and q — in decimal notation, 
say — can be done in two steps: 
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(i) First, multiply p and q as polynomial expressions: 
P = QnAn—1 +++ 149 = an 10" + dn_110""1 +--+ +4110 4 ao, 
qd= bnbn—1 sgt by bo = b,10” + bn—110"-! Se b, 10 + bo, 
pq = Con 102” + Con—110?"~-1 +» +1104 cp. 

(ii) Then, renormalize pq in correct decimal notation: 
Pd = Con+1€2n oe -C1Co, with 0 < Cj < 9, 0<i1<2n+1. 


Observe that only the first step is costly. But we have already seen how poly- 
nomial multiplication becomes circular convolution. 


Exercises 


(1) With the help of the Convolution Theorem, solve the equation a * x = b 
in the two following cases: 


0 —2 
1 —2 
(a) a= 9 | b= 9 
3 2 
1 2 
2 —2 
(b) a= 3 | b= 5 
4 2 


(2) Decide, with the aid of the Convolution Theorem, if the circular matrix 
C(a) with 


14+ 72 
mei 
samen Deer) 
-V/2 


is invertible. In the affirmative case, find the inverse matrix. 
(3) Show that 4F>1(u*v) = Fy'(u)Fy*(v) (use the identity F71(z) = 
(4) Define the inner product (,) :C” x C” —> C” by (x,y) = 42° -7. 
(Attention: (y,x) = (a,y) and (x, ay) = a(x, y) for a € C). 
(a) The nm columns fo, f,...,f,-1 of the matrix F,, are an orthonormal 
basis for this inner product: 


= 1 fori = J 
(fi, f)) = { 0 otherwise. 


(b) 4 (F(z), Faly ) a (x, ¥), Le. (Fr(x),y) = (a, F(y)). 
(c) (Fr(x *y),2) = (Fr(z), Py * Fa(2))). 
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3.1.2 The Fast Fourier Transform Algorithm 


The Fast Fourier Transform (FFT) is not an autonomous transformation. It 
is only a recursive algorithm which allows the fast computation of the vector 
y = F,2 for n = 2™ — i.e. whenever n is a power of 2— by iterated half-length 
reduction of the current vectors (a cascade visualizing in a binary tree). 


The Splitting Theorem 


Recall The recursive structure of the Pascal triangle for the binomial co- 
n—-1 n—-1 


efficients is given by the relation C) = (a) + ( k ) (in older notation: 
Ch= Cnt + Cha): 
We search a similar relation which allows, for n = 2m, to compute the vector 


y = F,x by means of the two vectors y’ = Fy,2’ and y” = F,,2", where v’ 
and x” are extracted from the vector x in a simple and direct manner. 


Fundamental observation: |w,, = we for n = 2m. 


Verify: every row (column) of even index of the matrix Fg depends only on 
W4. 


Whence the pivotal idea for a recursive algorithm. 
Reduce the computation of 


Yo vO 
Y1 =e 0 Ty 
Yn-1 En-1 
to the computation of 
Yo | v 
WY Fe v2 
Vrn=i T2m—-2 
and of 
Yo aca 
n 


Hypothesis Suppose to be already computed. 
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We would like to compute 


Yo 
Y1 
Yn-1 
in function of these two vectors. 
Let us write explicitly: 
y, = ye tapwe, OS jf <m-1, 
0<k<m-1 
Yj = yy Top wy, OS j<m-1 
0<k<m-1 
Yj = S- rw, O0<j<n-l. 
O<k<n-1 


Now we write once more y;, separating the even and the odd indices: 
= S 2kj S- 2k+1)j . 
Y= T2kWy Z + Torpaw ie 0 < J = n—1l. 
O0<k<m-1 O0<k<m-1 
Pie : 2kj kj 
But we = Wm, ie. w7) = wH, hence 
(= RI 4 yy i O<j< 1 
Y= L2RW yy + Wy: L2k41W 7% <jyeam-lil. 
0<k<m-1 0<k<m-1 
Finally: 


yy =y,tuh-yf, OS 5 <m-1. 


This gives formulas for the first half of the y-coordinates. We have still to 
find the formulas form <7 <n-1. 
We decide to use the relations wt? = —w/, and we obtain immediately 


Ymty = yj -—Wh yj, OS J <m-1. 
Splitting Theorem 


In the given situation, the components of y = F,x are represented in function 
of the components of y’ = F,,2’ and of y” = F,,2” by the formulas 


y=yj,tui-yf O<j<m-1, 


Ymn+j = Yi — wh yf OSG Sm—-1. 


3.1 The Discrete Fourier Transform 


The Algorithm for n = 8 


Yo xo 
Computation of wl] = Fg a 
U7 x7 
Yo ro Yo ry 
/ A 
Vy v2 YW X3 
1) If =F =F, 
( ) Yo ‘ 4 Yo . x5 
Y3 6 Y3 x7 
then we get: 
Yo=Yo+ Yo: Y4 = Yo — Yor 
W=HMtes yi, Y=yr—ws- yi, 
Y2 = Yo —i- ys, Ye =Yoti-ye, 
yx=Yygtus-yf, yr=yy—we-y8 


(2) In order to compute 


Yo Ba) 
#1 = Fy a ry 
Y2 v4 
¥3 x6 
we need : 
= Xv 
(eae 
and 


We shall get 


Yo=YotYo, ye=Yor- VY 
y=y,—i-ylt, ys=y, ti- yf. 


In order to compute 


Yo Ty 
ie poe Fy “ ’ 
Y2 x5 
¥3 U7 


we need 
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and 


(3) Finally, in order to compute i = Fy & i: we need yj = Fi xo = Xo 
1 4 


and yj = Fia4 = £4, and we obtain 


Yo=¥Yo+tY, Yi=Y—Yo- 


Similar for 


and 


Note that Fy = ; = , in perfect harmony with our formulas. 
Scheme for the computation of 
Yo vO 
eae (oe Fe v1 
Y7 z7 


Yo |Y1 | Y2|Y3 | U4 | Y5| Ye |Y7 
/ / / / Wi W " WW 


Yo} Y1|¥2 | Y¥3|Yo |Y1 |Y2 |Y3 
Yo | Y1 | Y2|Y3 1 Yo | Y1 | Y2|Y3 


/ / 1), MW), Tt / WM), 


Yo] Y11Yo1Y1 1 Yo} 911 Yo |Y1 
Yo|Y1 | Yo|¥1} Yo} ¥1)Yo\Y1 


/ MW) MW), WW), i 


Yo|Yo|¥o|Yo| Yo |Yo}Y¥o|Yo 
LQ| C4 | V2 | VE | Ly U5 | U3 |V7 


3 
—i 
al 
1] i 
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1/0|0/0] 1 | 0} 1 0 
1)0|5]0} 0 | 0 |—40 
3 1 i i 

4|9)7/9)—a) 9 | g |0 
3/3/1)1}_ if_oai} i ji 
818)/8]8 8 8] 8 {8 


Exercises 


(1) Compute, via the FFT scheme, y = Fx for 


(2) 


(3) 


(4) 


(5) 


(6) 


0 1 
1 2 

5 8 

a ey lee = I 
0 4 

i 3 

1 2 


Let A = C(a) be the circular matrix the first column a of which is given 
by ao a, = 0, ag a3 = 0, a4 as = 0, ag = —4, a7 = 0. 


1 1 1 

Using the Contalisinr T ficorem show that the matrix C(a) is cwehabie. 
and compute C(a)~?. 

You dispose of a program FFT-256 that computes y = F562. How to use 
this program in order to compute x = F, none 

Let A = C(a) be the circular matrix the first column a of which is given 
ao = 3, a = 5, a2 = §, 3 = —Z - 5 2 tie = 0Gy 4, Oe = 0, 
a7 = —+ + ; 2. 

Using the Convolution Theorem, show that the matrix C(a) is invertible, 
and compute O(x) = C(a)7?. 

Compute ;% Fi6(«) for « € C'® with 


xo = 2, gy =1lt+vV2+i, v2 =2V242i, 23 =1-V2-i, 
ta=24+4i, v5 =1—V2+4+i, we =—-2V2-2i, z7=14+V2-i, 
ag = 2, fg =1+V24+i, xy = -2V24+ 2%, ay, =1- ¥2-i, 


tq =2—4i, 213 =1—-V2+i, a4 =2V2-2i, 1p =14+ V2-i. 
Compute 7 Fi6(«) for « € C'® with 


ro=10, y= 1430+ V72)i, c= —2, 23 =1-—3(1— V2); 
t4=-2, e=1+3(1—/2)i, 2g =-2, 27 =1—3(1+ V2), 
tg=—6, t9=14+3(1+V72)i, ei =-2, 21 = 1—3(1-— V2)i, 
tq = —2, %13=14+3(1— V2)i, a4 = -2, 215 = 1 — 3(1 + V2)i. 
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Execution of the Algorithm FFT 


We shall adopt a row-notation. Let n = 2'. Consider first the binary tree of 


the input vectors: x = (,) = (0, 21,---,£n—1) will be sorted as follows: 
x(0) — (ax), 
x40) = (94) x0) = (ap 4), 
x(2,0) — (Tak) xO (Tan+2) X 72) — (441) x(2:3) = (Tan+3), 
x30 98D xB xB yBHN. BH. x36) x(3.7) 


(age) (tskt+4) (@8k+2) (esk+6) (@sk+1) (#8k4+5) (@8r4+3) (*8k+7)- 


The jth row, 0 < j < 1, consists of 2’ vectors, of length 2'~ each. We will 
have, in general: x”) = Callen eee ae with 0 < j <1 (j is the index of 
the current row) and 0 < m < 2/ — 1 (m is the index of the block k in the 


current row). Note the recursion formulas: 


(0,0) 


ry = Xk, 
(j+1,2m) _ (,m) 
Uy = on > 
(f+1,2m+1) _ , (j,m) 
Ly = opti: 


Notation: x[q] := ), 


We shall try to find the index g = q(j,m, k) such that ohm) = a[q(j,m, k)] 
(“the kth component of the mth block in the jth row is 2[?]”). 


Example Distribution of the indices for / = 3. 
000 001 010 011 100 101 110 111, 
000 010 100 110|001 011 101 111, 
000 100|010 110]001 101|011 111, 
000]100|010]110|001/101]011\111. 


On the jth row, the l-j first digits enumerate (in binary notation) the com- 
ponents, the j last digits enumerate the blocks. But the enumeration of the 
blocks is not monotone. 


Definition The function “word inversion” : 
p;(Bj—-129 + 8j—22?-7-+- --+-912+- Bo) = (0027-1 +6129 -7-+- --+8j-22+8;-1). 


Example p5(11001) = 10011. 
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Recursion Formulas 
pj+i(2m) = pj(m),  py4i(2m+1) = p;(m) + 2, O0<m<2-1. 


Lemma q(j,m,k) = 27 -k + p;(m), 
ie. 22” = a2 -k+p;(m)],0<j7<L0<m<2%-1,0<k< 2-3-1, 
In particular, we get for the singletons of the last row: 
lym 
ay” = alpi(m)]. 


Proof Recursion on 7 (Exercise). 


Example Let x = (%0,%1,---,1,023) be a vector of 2'° components. Which 
component of x will occupy the 313th place of the last row of the input-tree 
for the algorithm FFT? 

Answer: x") = a[p19(0100111001)] = x[1001110010] = xe26. 

Consider now the binary tree of the Fourier transforms (with yim) — 
Fy—jxb™); 


y (09) 


y (9) yb) 


y (2:0) y 2) y (2?) y (2:3) 


y 3:9) yb y 3?) y 33) y 34) y 35) y 39) y 27) 


y 9) = Fx) is the final result. 

We know the row of the basis-singletons: 

ym) = yf = af") = wlpi(m)). 

The vectors of the intermediate lines are computed by the Splitting 
Theorem. 

Put rj = Wo; = e7?7/? | Then 


j—1, j,2 j,2m+1 
yl sm) _ yo TD eh ge yh m-+1) 


j—1,m j,2m j,2m+1 
Te = : ) Pee -y¥ 
Consider every row of our tree as a single vector of length n = oF 
Now write yj; = (y;[k]), ie. yj[2’% -m+k] = yo, 0<m< 2-1, 
0 < k < 2'-J —1 (we proceed horizontally, block after block). 
Recall that we have, on the level of the singletons: 
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nlm] = a[p(m)], OSms 2-1. 
With this new notation, the splitting equations become 
yj—1[2 9 m+ kl = yj[2" 9 mt Rl tert yyy 2? OP mt ket 2), 
yj;—1[2° 9 -m+k4+2'-3] = Yj [2!'-3+1 om + ki] 
- ee -y;[2°-F1 emt k+2°-4. 


Put h:= 2'-+1.m+k, s:= 2!-3, 
Auxiliary observation: rf_j., = 7/41 (h = k mod 2'~J+!). Whence our 
recursion formulas in final form: 


y5—-11h] = yglh] + rp ya wilh t+ 5] 
yj—i[h + 8] = ys[h] — ry j41  uylh + 5] 
for 7 =1,f—1,...,1, 6 =2'-J, h= 2-311 -m +k, 
with m =0,1,...,27-' -l and k =0,1,...,s—1. 


Conclusion If the vectors y; = (y;[h]) are computed according to the re- 
cursion formulas above, then the yo[m], 0 < m < 2! — 1, are the components 
of y = Fx. 


Exercises 
(1) Let y = Fx. 


True or false: the FFT scheme, applied to an input vector x, the components 
of which are ranged in natural order, produces the transformed vector y in 
p3-permuted order: 


Yo |Y4 | Y2| Yeo} U1 | Ys | Y3 | U7 


L/L |L2|V3\L4|V5|LE|L7 


Tv 
(2) Letx= | - | bea vector with real components, y = Fx. Show that yo 
x7 
and yq will be real, and that y, = 97, yo = ye and y3 = J. 


3.2 Trigonometric Interpolation 


In this section, we shall proceed to give a more conceptual interpretation of 
the Discrete Fourier Transform, in the language of signal theory. 

More precisely, we shall try to clarify the mathematical meaning of the 
statement: the Fourier transform passes from the time domain to the frequency 
domain. 
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Despite a thunderous horizon of practical questions, our presentation will 
always remain naively mathematical; for example, we shall never touch upon 
the problem of digitization, i.e. of effective sampling. 


3.2.1 Trigonometric Polynomials 


Consider (complex-valued) trigonometric polynomials, in real notation, i.e. of 
the form 

f(t) = G + a,-cos2nt + by - sin2rt + ag-cosdrt + bg-sindnt +... dm: 
cos27mt + bm - sin2amt 

with ao,@1,.--,@m,61,...,6m € C. 


Remarks 


(1) f(t) is a function of the real variable ¢ (t = the time), taking complex 
values. 

(2) The time t is normalized: f(t + 1) = f(t) for every t € R. 
f(t) is thus a 1-periodic function; in another language: f(t) is an elemen- 
tary 1-periodic signal. 

(3) Example: 
f(t) = 3+ (2+i) cos 2nt — 3 sin 27t —icos 4at — (1—i) sin dat = x(t) +iy(t) 
with 
x(t) = 3+ 2cos2zrt — 3sin 2rt — sin 4rt, 
y(t) = cos 2nt — cos4nt + sin 4rt. 


We observe: our formal generosity to admit complex coefficients means con- 
ceptually that we consider two periodic real signals x(t) and y(t) (the real part 
and the imaginary part of f(t)) simultaneously. You should imagine their si- 
multaneous progression in two orthogonal planes. 

Let us insist. The factor 7 serves merely as a separator: our 3D coordinate 
system is composed of the taxis, together with the z-axis and the y-axis of 
the usual complex plane (orthogonal to the t-axis). Multiplication by 7 means 
a rotation of 90° around the t-axis (note that it has absolutely no phase shift 
signification in the current formal context). 

Concerning the language. The notation of f(t) is called real, since f(t) = 
x(t) +iy(t) is given explicitly by two real functions, linear combinations of the 
elementary functions cos27vt and sin2avt, O<u<m. 


Let us now pass to a purely complex notation. 


Recall 
e? = cos@ +isin# 


hence cos @ = $(e® + e~ #9) and sind = —4 (el? — e7 9). 
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We obtain 
f®= > + a,-cos2rt + b1- sin2nt + az+cos4at + bo- sindnt 
+... Gm + cos2rmt + bm + sin2amt = 0 + PS Gv (g2mivt ae 
2 2 
l<v<m 
ns Se et = eure) = > > 5 (ay ibjjen 
l<v<m l<v<m 
1 : . 
25 S- 5 (wv of ig, jee = S- eer 
l<v<m —m<v<m 


with the transfer formulas between the real notation and the complex no- 
tation: 
Cy = $(ay — iby), cp = $(a, + ib,), O<v<m, 
Qp=C+cyr, bb =i(cy—c_p), O<u<m. 


(Note that the negative frequencies occurring in the complex notation have 
no conceptual interpretation and disappear when passing to the real notation.) 


We shall call f(t) = 0 me<v<m cye2""t a trigonometric polynomial 
(in complex notation) of degree m. 


And indeed: f(t) = c_mX7™ + ¢ om4iiX tt +c 1X +e9 +X + 
eee bm 1X! bey X™ with X = e27%, 


Exercises 


(1). Let f(t) = be-8r# + 4 4. 4 Le Bat, 


Find the real notation of f(t) and compute the values of f(t) for t = 

(2) Let f(t) = 4-— 2cos2rt + 3sin 4at — cos 6rt + 2sin 6rt. 

Write f(t) in complex notation f(t) = 032,23 wer". 

(3) het FO) Hs aoe ese Be a, trigonometric polynomial of degree 
m, in complex notation. How do you recognize in this notation that f(t) 
is actually a real function? 

(4) Let f(t) = 2—cos2nt + 3sin2nt + 4cos4rt — 2sin 6at + cos 8rt. 


Find the complex notation of f(t) = f(t — 4) = @se~8™# + ge Ott + 
E_ge~ 47 4 &_ ye 2H + hy + Ger + Get 4 GyeO™# 4 Ge®™!, ie. find the 
coefficients ¢, for —4<y< 4. 
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3.2.2 Sampling and Reconstruction 


In the following pages we shall promote a Sampling Theorem which is the 
purely discrete periodic version of a celebrated theorem (associated with the 
name of Shannon), which we shall encounter in a moment. From an unpreten- 
tious mathematical viewpoint, we are simply in a situation of (trigonometric) 
interpolation. 

Consider, as a first step, for v € Z, the elementary 1-periodic signal 


ell (t) =e?!" = cos 2Qavt + isin 2nvt 


(geometrically, this is one of the possible parametrizations of the unit circle). 
Now, for n > 2, carry out a sampling of order n, i.e. consider the n values 

el], el], ee ei. where el] = el¥l() SeFrMwyh Sag’ VS kw — 1, 
We observe: the vector 


of these n samples (for t = 0, 1, 2. Seed n—t) is one of the columns of the 
matrix Fy. 

More precisely: let v9 be the (non-negative) remainder of division of v 
by n (vp = v mod n), then we are confronted with the voth column of Fy! 


(0<um<n-1). 


Conclusion 
el] 0 
el] : 
—F, . =e, =| 1] <— position. 
n 
an 


Let us make this explicit for n = 8: 


el? ell 
0 0 
0) 1 1 0 
‘b 0 ‘i 
1 
€5 0 C5 0 
0 1 
1 e: 0 1 e: 0 
— Fe 3) =€0 = , — Fs at SH 41 = > 
€4 0 8 €4 0 
0) 1 
ex 0 es 0 
0 0 1 0 
€6 &6 
0) 0 1 0 
e7 er 
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el? el3 
9, 0 9, 0 
ee 0 oh 0 
es 1 €5 0 
ra Ee) eee ea ag ee 
8 8 el? = £2: 0 ’ 8 8 el} = 3 = 0 ’ 
ee 0 ee? 0 
io 0 73 0 
ee e 
6, 0 6, 0 
e7 e7 
elt el-4 el 3 
9, 7 0 Br 0 
7 co 0 oe 0 
4, ‘ 4 ae 3 7 
1 €3 1 €3. _. _ | 0 1 €3. eo en iO 
gts el = gts el 4 ie 1]? gts ef, * aaa Om 
“ oF ° ae ; 
e e en 
6, 6 0 Boe 0 
e€; ey ey 
él? eft 
i= 0 et 0 
1s 0 1 0 
€5 0 e€5 0 
1 eb? 0 1 elt 0 
“fF 3 ae =P 3 oe 
8 ; ef? FS O}]’ 8 : e, re 0 
ele 0 ef 0 
—2 1 —1 0 
6 6 
el? 7 et} 1 


7 
Let now f(t) = >_,,epem wer’ be a trigonometric polynomial of degree 
m, in complex notation. — 

We sample to the order n = 2m: 


fo= 10). f= 1 (2) =F (2) ster =s (>). 


Now 


= 


hence we have 
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As a consequence 


fo “8, Co 
v 

fi e+ C1 

Cm-1 

n = 
—m<v<m i Le ue 

C_m+1 

= [v] om 

Foi 1 Cred 1 


In order to get rid of the nasty and ambiguous central term c,+C_m, observe 
the following simple fact. 

A sampling to the order n = 2m of 

f(t) = B + a, -cos2rt + b+ sin2at + az-cos4nt + bz: sindnt + ... dm: 
cos2mmt + b,, - sin2amt 

is perfectly insensible to the value of b,: sin Qn mE = sinkz = 0 for 
O0<k<n-1. 

In other words, it is impossible to recover the contribution of the term 
bm -sin2amt after a sampling to the order n = 2m. 

With unicity of reconstruction in mind, we shall limit our attention to 
balanced trigonometric polynomials, i.e. to trigonometric polynomials of the 
form 


fo= > + a,:cos2rt + by-sin2rt + ag-cos4nt + bo-sindnt+... a -cos2rmt 
(hence with b,,, = 0). This is equivalent to a complex notation of the form 


(a een ee . Cm . 
f(t) = 5 e arimt : cpermivt + 5 e2mimt 
—m+l<v<m-1 


with cy = C_m. Note that we have still dm = Cm +C—m (this is the reason 
of the factor $*). 

Why is the factor 4 appended to the extremal terms of the complex nota- 
tion? 

The reason is simply the pleasure of a formal harmony: we want the central 
term of the transformed vector to be called cm (= cm) and not 2¢m(= 
Cm + Cm): 


Remark Note that we already have convincing arguments for the Discrete 
Fourier Transform to pass from the time domain to the frequency domain. 
The input vectors are time-indexed, i.e. they are (naturally extensible to) 
periodic time series, the output vectors are frequency-indexed lists of (eventu- 
ally cumulative) amplitudes for interpolating trigonometric polynomials. Let 
us sum up: 
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Sampling Theorem 


C_m _—2ri 4 Cm i 
f(t) = 5 e anime : cpermive + 5 e2mimt 
—m+l<v<m-1 


a balanced trigonometric polynomial of degree m. 
We sample to the order n = 2m ?: 


Then 

fo Co 
fi C1 
Cm— 

=p = m—1 

n Cm = C_m 

C_m+1 

fn-1 C1 


Note that our Sampling Theorem can also be read as a 
Trigonometric Interpolation Theorem in the following manner: 


Let fo, fi,---)fn—1 be n = 2m complex values. 

Then there exists a unique balanced trigonometric polynomial of degree 
<m 

f(t) = B + a,-cos2rt + b,- sin2at + az-cos4nt + be-sindnt +... &- 
cos27mt 


with f(*) = f,,0<k<n-1. 


FFT scheme for trigonometric interpolation in case n = 8: 


Co Cy C2 C3 C4 | C_3 | C_gq | C_y1 


FOMFOFOFOFO FO FO FQ 


? Note that our reconstruction works under the assumption that the sampling fre- 
quency n is twice the maximal frequency m in the (naive) spectrum of f(t). This 
will be Ariadne’s thread for Sampling Theorems. 


3.2 Trigonometric Interpolation 197 


Exercises 


(1) 


(2 


a 


(3) 


(4) 


Find the balanced trigonometric polynomial f(t) = 4 + a1-cos2at + by- 

sin2nt + a2-cos4nt + by-sindrt + a3-cos6rt + b3-sin6rt + G-cos8rt, 

such that f(0) = 1, f(%) =2, f (4) =3, f (2) =4, f (5) =5, f (8) =4 

£(3) =3, FB) =2. 

Which is the balanced trigonometric polynomial f(t) = 4 + a1-cos2at + 

by -sin2rt + ag-cos4nrt + bo-sindrt + a3-cos6rt + b3-sin6rt + i -cos81t 

with f(0) = f (3) =1, f(a) = f(s) =-1, F(a) =F) = 1 FG) = 

f(f)=-2 

Find the eight balanced trigonometric polynomials of degree 4 fo(t), f1(t), 
.., f7(t), which sample (to the order 8) into the eight unit vectors 

€0,€1,-+-,€7- 

Transmission of a block of 16 bits (two 8-bit bytes) by frequency modu- 

lation. 


XY = %0%1 °°: &7yoyi +++ y7 will be transmitted by means of 


f®H= > + a,-cos2rt + b, + sin2nt + ag-cos4nt + bo- sin4nt 


+a3-cos6nt + b3- sin6at + ee cos8rt 


with 


ao = Xo + iyo, 

ay = M1 iyi, by = %4 + iya, 
a2 = 22+ iyo, be=25 + iYys, 
a3 =%3+iy3, 63 = 6 + iYe. 


a F 
oe = &7 + 1Y7, 


After sampling f(t) to the order 8, it is possible to reconstruct XY with 


the help of the Discrete Fourier Transform. 


‘You sample: 


f(0) =2+-4i, f () =1+ V2 +i, F(Z) =0, f (3) =14+ V2 +i, 
FG) =2 £ (3) =1-v2+i, f (9) =0, f (§) =1- v2 +i. 
Find XY, i.e. the 16 transmitted bits. 
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3.3 The Whittaker-Shannon Theorem 


In order to better understand (and perhaps to appreciate) our modest Sam- 
pling Theorem in the elementary periodic case, let us now face the sacrosanct 
result for all arguments concerning the transition analog — digital — analog: 
the theorem of Nyquist-Shannon (as it is ordinarily called in engineering 
textbooks; let us nevertheless point out that the priority is Whittaker’s, who 
proved it in 1935, 20 years before Shannon). 


3.3.1 Fourier Series 


In the core of this section, we will be confronted with the following inter- 
polation problem: what type of function (of signal) is determined by an 
appropriate sequence of its samples? 

There is a similar problem, that of developing a periodic (piecewise contin- 
uous) function into a Fourier series. In this case, the sequence of the Fourier 
coefficients determines (the form of) the periodic function. 

Actually, these two problems are intimately linked — we shall see this in a 
moment. 

Let us begin with a brief summary of Fourier series. 


The Situation — Temporal Version*: 


Let f(@) be a function of the real variable 6, with (real or) complex values, 
and periodic, of period P = 2W. 

Put T= Ww: 

(2rT will be the pulsation associated with the period P = 2W). 

In order to begin decently, we shall suppose f(@) to be continuous (even- 
tually with a finite number of simple discontinuities in the period interval). 

Consider the sequence of the (complex-valued) Fourier coefficients of 


(9): 
Ck = ne / i f (Oe FTP dd, ke Z. 
2W J_w 


These coefficients — arguably somehow enigmatic for the novice — are nat- 
urally obtained as the solutions of an extremal problem: 

The trigonometric polynomials 

on f (9) = o_nepen Cee” 

are, for every value of N > 0, the best approximation of f(@) in the follow- 
ing sense: 

Consider, for N fixed, all P-periodic trigonometric polynomials 


p(0) = S- pert? 


—N<k<N 
the degree of which does not exceed N. 


3 Fourier series occupy two sensibly different places in mathematical signal theory. 
First, they formalize correctly the time domain analysis of periodic phenomena; 
then, they introduce “spectral periodicity” as Fourier transforms of time series. 
It is the first aspect which they were invented for — but it is the second aspect 
which guarantees their mathematical interest. 
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Measure the distance between f(@) and each p(@) by the “energy-norm” 
of their difference: 


w 
I) — (@)| = i i |, WO) — w(o)reae. 


Then each oy f(@) is the trigonometric polynomial of degree < N which 
minimizes this “energy-distance” . 
Let us insist: 


on f (0) = > Cherie? 
—N<k<N 
best approximates (replaces) the function f(0) to the order N for a spe- 
cific notion of distance between functions (one computes a certain separating 
area). Note that a priori oy f(@) is not attached to any interpolation problem. 
Pass now to the trigonometric series 


of(@) = lim ow f(8) = >) cxe?™*7. 
keZ 
The fundamental question is then the following: 


What is the relation between the periodic function f(6) 
and its Fourier series of (0)? 


The answer is given by the Theorem of Dirichlet, which reads in small talk 
version like this. 

If f(@) ts sufficiently regular, then the series of (0) will converge for every 
6 ER, and will represent the function f(@): 


f(9) = of @) = Sj cxermihr?. 
keZ 
Let us underline: In this case, f(@) is entirely determined by the sequence 
(cz) kez of its Fourier coefficients. Remains to clarify the condition of regularity 
on f(@), which forces the equality f(@) = of(). 


(A) Weak regularity. f(0) is of bounded variation in the period interval. 
Consider, for every finite partition P = (—W = 0 < 61 <--- <6, =W) 
of the period interval, the sum 


VP)= YS) |FOr+1) — F(9x)I- 


0<k<n-1 


If the set {V(P)} of all these values admits a finite upper bound, then 
f(@) is said to be of bounded variation in [—W, W]. 
In particular, this is the case for differentiable f(@), admitting a bounded 
derivation f’(@). 

(B) Strong regularity. If f(@) is twice continuously differentiable, or if the 
series )°,.<z Ck is absolutely convergent, then o f(#) converges absolutely 
and uniformly to f(6). 
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Exercises 


We shall consider only real 27 — periodic functions (i.e. with W = 7), which 
will be specified on an interval of length 27. It is clear that the transit formu- 
las between complex notation and real notation introduced for trigonometric 
polynomials are the same for trigonometrical series: 


0 for0 <0<17, 
Gy TG) 2n —@ for 7 <0 < 27. 
Show that 
nan A cos3@  cos5@ ~~ cos7é 
of(0) = 5 * (cosa + 32 + 52 + 7 +e), 


0 for —-7 <0 <0, 
(2) f(0) = 0 for0 <@< $, 
ap lok S & Oa 


Find of (0). 
(3) f(@) =0 for -7 <0< 7. 
Show that 


sin20 sin3@ sin4é 
2 3 4 , 


of(0) =2 (sind en 


(4) f(0) = 6? for -7 <0< 7. 
Show that 


Wr cos20  cos3@ cos 40 
af(0) = = —4 (cost 52+ 32 Bt oy). 
Fourier Series — Spectral Version 


The development of a periodic function into a Fourier series is a standard 
modelling method in engineering (of acoustic, electroacoustic and other wave- 
dominated phenomena). 

On the other hand, mathematical signal theory uses Fourier series almost 
exclusively as support for formal information, i.e. in the frequency domain: 
they appear as the Fourier transforms of certain time series (i.e. of discrete- 
time waveforms). We shall see this when following the arguments of the proof 
of the Nyquist-Shannon theorem. 

From a formal viewpoint, the most natural presentation of the subject 
emerges (as often in Mathematics) from a geometrization of the situation. 
This makes the Hilbert spaces of square summable sequences and of (classes 
of) square integrable functions appear. 
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Recall 


(1) I?(Z) is the space of (complex-valued) sequences ¢ = (cx) ez such that 


Se lel? Ses: 


keZ 


The inner product. (c,c') = opez crc}, (conjugation of the second factor) 
The norm. |le||? = (c,¢) = Spez lex? 

The Cauchy-Schwarz inequality. (c,c’) < |le|] - |e’|| 

1?(Z) is complete for the associated uniform structure: every Cauchy se- 
quence in 1?(Z) converges (to an element of 1?(Z)). 


(2) L?[—7, 7] is the space of (classes of) functions which are square integrable 
over the interval ] — 7,7]. 


More precisely, a (complex-valued) function f(w) of the real variable w, 
—m1 <w <7, is square integrable over | — 7,7] if 


J \fwyPaw <0 
(i.e. the relevant Lebesgue integral is defined and takes a finite value). 
Consider the equivalence relation. 


fw) ~ gw): 
=> f and g are equal almost everywhere 
<> {w: f(w) 4 G(w)} is a set of zero measure. 


L?|—7, 7] will then be the space of square integrable functions on | — 7, 7], 
identified according to the relation of almost everywhere (a.e.) equality. 

In the sequel, we shall always write functions — but think in terms of 
classes of functions. 


Hence: 

The inner product. (f,g) = ad " fw )g(w)d 

The associated norm. ||f||? = (f, f) = ayn = w)|?dw 
The Cauchy-Schwarz ee (f,9) < | zn aa 


L?[—7, 7] is complete, exactly like 1?(Z). 


But an indispensable remark concerning the convergence in L?[—7, 7]: 


f= lm f, <= lim iE fo (w)|?dw = 0 


(convergence in quadratic mean). 

Conceptually, we deal more or less with a convergence in shape: a sepa- 
rating area becomes smaller and smaller, hence the line of f(w) is “generi- 
cally” (almost everywhere) approached. But attention: a priori, convergence 
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in quadratic mean does not involve (almost everywhere) simple convergence 
(an element of L?[—7, 7] has no values — it has a shape). 

Note, moreover, that L?[—7,7] is, from another viewpoint, the space of 
(classes of) 27-periodic, locally square integrable functions: ]—7, 7] will figure 
as the standard period interval of length 27. 

Let us now pass to the formalism which identifies 1?(Z) as coordinate space 
with respect to the classical orthonormal basis of L?[—7, 7]. 

For f (w), square integrable on |—7, 7], we define the sequence of its Fourier 
coefficients as follows: 

f ; 1 off Shek 
flk] = (fw),e*) = = J fwel*dw, keZ. 
27 Jin 

[We note a friction with the time domain formalism: a natural change of 
variable w = 27T@ yields a sequence of Fourier coefficients enumerated in the 
opposite sense — our frequency domain definition will finally be justified by 
its harmonizing with the formulary of the inverse Fourier transform. ] 

Then we get the following fundamental result: 


The correspondence f(w) +> (f[k])xez 
induces an isomorphism of Hilbert spaces between L?[—7, 7] and 1?(Z). 


More precisely: 


(1) If f € L?[—z, x], then the series of its Fourier coefficients is square sum- 


mable, and 


2s, 1 ce 2 
Si lfalP= =f lw)Paw 


kEZ ei 
(Plancherel Formula). More generally, if f, 9 € L2[—7, 7], then 


Sse = 5 f° fw\ioaw 


kez 


(Parseval Identity). The Fourier series of f converges in quadratic mean 
to f: 
2 

dw = 0. 


> file * — fw) 


In other words, {e~”*, k € Z} is an orthonormal basis of L?|—r, 7].4 

(2) Conversely, if (f[k])xez is such that >.<, |f[k]|? < oo, then the series 
Dnez f[kle7'** converges in quadratic mean to an element f €L2[-x, 7m], 
and f is the only element of L2[—z, 7] which admits the sequence f|k], 
k € Z, as the sequence of its Fourier coefficients. 


4 Le. the linear span of the given orthonormal system is dense in L?[—7, 7]. 
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There remains nevertheless the question: which is the result that replaces the 
Dirichlet theorem in this context? 
The best imaginable statement should be the following: 


If f € L?[—7z, 7], then the Fourier series of f converges almost everywhere. 


This is in fact a theorem which has been proved by Carleson in 1966 (!: 
the memoir of Dirichlet on the convergence of Fourier series dates from 1829). 


Remark Do not forget that non-constructive results which are true “almost 
everywhere” may be empty from the algorithmic viewpoint: the field of the 
real numbers which are effectively computable is of measure zero! 


3.3.2 The Whittaker-Shannon Theorem for Elementary Periodic 
Functions 


We shall try to reconstruct a simple periodic function 

x(t) = 2 + ay -cos2rt + by - sin2nt + az-cosdrt + bz: sindnt +... dm 
cos2r7mt + bm + sin2amt 
(i.e. a trigonometric polynomial of degree m) with the help of an equidistant 
sampling to the order n > 2m (n = 2m in the balanced case), without passing 
by the frequency domain. The solution will be a little mathematical perversity 
— we shall have approached something simple by something complicated — 
but at the same time we will have constructed a bridge between the Discrete 
Fourier Transform landscape and the highlands of Whittaker-Shannon. 

Let us begin with an (elementary, but non-trivial) result of Calculus: 


Recall Development of certain trigonometric functions into series of simple 
fractions. 


(1) SCT) a = eee 1)* (ck + sr); 
(2) -cot(n8) = § + Duss (abe + zhp) for every 6 ¢ Z. 


[Note that both series are absolutely convergent: for example, the second 
satisfies, for k > 2|6]: 


1 1 2|0| 8|0| 
= < P 
O0+k O-k k? — 6? ~ 3k? 
which permits us to obtain an estimation of the significant remainder by 
8|9 
Wy 2 
This yields the following identities, valid for every 6 € R: 
(1) 1= Daex(-1)* Sem 
sin(70) 
(2) cos(7) = inex = (0— ee 
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But, cos(k7) = (—1)", which gives finally: 


sin 7(0—k 
(1) 1= Deer Soe 


(2) cos(18) = Tpen, cos(kr) - eae W, 


In order to obtain a formally well-established expression, let us replace 6 by 
2Wt, and put T = x7. 
This yields: 


sin 2nW (t—kT 
(1) 1= Diner Sara 


(2) cos27Wt = 91 ,¢7.cos(2nW (kT)) - VGC 


We insist: 
Let x(t) = 1 or x(t) = cos27Wt, then 
as sin 2nW (t — kT) 
Tk’ OnW(t—kT) ” 


keZ 


where x, = (kT), k € Z with T = xp. 
An elementary signal of the form a(t) = 4 + a1 - cos27W¢t can thus be 
reconstructed, after equidistant (infinite) porter by means of the sequence 
(ty = X(kT))pez of its samples, in form of the series 


=. sin 2nW (t — kT) 
Lp 


QnW(t—kT) ’ 
keZ 


where T = 5}; (W is the frequency of x(t). 
This is precisely the theorem of Whittaker-Shannon for a signal of the 


form 


x(t) = 5 + a, -cos2nWt. 
Note that at this rather primitive stage, the sequence of the samples is 
2-periodic. 
Clearly, this innocent result can be generalized. We will show the 


Nyquist-Shannon theorem — the elementary periodic case.° 


Let 
a(t) = > + a;-cos2rt + b1-sin2rt + ag:cos4rt + bo-sindnt +. oe -cos2rmt 


be a balanced trigonometric polynomial of degree m. 
We sample to the order n = 2m: x, = x(kT) with T = + = = (the 
sequence (a%)kez is n-periodic). Then 


ZS; sin 2rm(t — kT) 
Lk 


Qrm(t—kT) © 
keZ 


5 Despite Whittaker’s priority, let us occasionally respect the traditional labelling. 
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Proof First, note that we have normalized the time, as we did often in the 
preceding sections. 


We shall begin with the case of a trigonometric polynomial which is an even 
function (i.e. by = bg =--- = by, = 0). 


Auxiliary Result 


The function f(0) = cos(a@), a ¢ Z, admits for —7 < 6 < a the following 
development in a Fourier series: 


ane) = sin(a7) | 2a 


Qa OS 


Exchange now a and @: 


in(76) | 1 1 1 
ealanys Se) | (Gata) me 


1 1 
Qa—-:: 
+(ge5 + pay) 20 : 


sT(-1)* cos(ka)- > sin(70) | ae =e sae _sin 7(0 — k) 


keZ keZ m(O a k) 


Replace 6 by 2Wt and a by ® q™ 9 a <1; we get, with T = Ww 


(t) cos 2n (2 w) t= cos (2" (2 w) (Kr) ; Sane: 


keZ 


Let now 

x(t) = B + a1 cos 2nt + ag cos4rt + +++ + Am cos 2mmt 

be an even trigonometric polynomial of degree m. Put W = m, n = 2m, 
ie. T = +. Then we immediately obtain from ({) and by linearity of the 
sampling operation: 


27m x — £) 


=v 
Tk 
2mm ( 


keZ ae) 


where x; = x(#), k € Z. This is clearly the Nyquist-Shannon theorem for 
an even trigonometric polynomial. 

In order to get our theorem in full generality, we have still to show the 
odd case. 
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Exercise 


(a) Let g(@) = sin(a@), a ¢ Z, and let og(@) = limn ong(9) be its Fourier 
series. Show that 


og(0) = -S(-1)* (4 <i) - sin(k6). 


k>1 
We shall admit that g(0) = og(0) for -77 <0 <7. 
(b) Deduce that we get, for x(t) = sin 27 (ew) twithO<*% <landT= ww 


= sin 27W (t — kT) 
oe ae. “OnW(t—kT) ” 


where ry = 2(kT), k € Z. 


Combining the result of the exercise with our previous result in the even case, 
we finally obtain the theorem as stated. 


Note that the reconstruction of the function sin27Wt by a sampling with 
step size T’ = nia is not possible: sin(kw) = 0 for k € Z. This explains the 
appearance of the balanced form in the statement of the theorem. 


Important Remark 


The theorem of Nyquist-Shannon — elementary periodic case — as stated 
above, is an equivalent version of our Sampling Theorem of the preceding 
section. 

More precisely: let n = 2m. Then, for every n-periodic sequence (2%) ez 
of (real or) complex numbers, the series of functions 


sin 2rm(t — EY 
a(t) = Cae a 
pase aarm(t— =) 


is convergent for every t € R, and it equals the balanced trigonometric poly- 
nomial 


Cm _97; A Cm ‘ 
x(t) = 5 e 2rimt Ae S cper™vt ae ae 
—m+l<v<m-l1 
where 
Co Ba) 
Cy Ty 
Cm—-1 
=> —F, 
Cm = C-m n 
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Question Is it reasonable to compute the values of a trigonometric polyno- 
mial with the aid of the Nyquist-Shannon interpolator? 


Answer Consult the following exercises. 


Exercises 


(1) Let x(t) = cos2mt. One knows: x (4) = cos 3 = $. 
sin 2n(t 5) 


(a) a(t) = Dpez(-1)* - a(t ~+ according to our foregoing results. 
TL 9 
Put . 
2m (t— = 
ry(t) = Cy ee 
—N<k<N 2m (t— 5) 
Show that Fe 
1 BVve pa 1 
ev(1)=™. (2-5 
(5) 7 (; L4 a1) 


and compute x4 (z), then ©10(%): 


(b) How many terms of the series above must we compute in order to 
obtain x (4) — $ with a two digit decimal precision? In other words, 
find N > 1 such that |5 — xn (4)| < q- 
(2) Let still x(t) = cos2at, but now sample to the order n = 8. 
We will have: 


eye sans k _sin 87 (t — §) 
One $2 (=) an (t— 5) 


Let xy(t) be the “truncated development of order N” of x(t), as in the 


preceding exercise. Compute va(a)s and compare with the result above. 


Summary Restricting our attention to elementary periodic signals, we ob- 
serve that the problems of sampling and of reconstruction (i.e. of interpola- 
tion) are sensibly better resolved by the methods of Linear Algebra — via the 
Discrete Fourier Transform (and the associated algorithm FFT) — than by 
Analysis: the Nyquist-Shannon interpolators converge very slowly, and are 
thus rather imprecise (clearly, you can always improve by oversampling — as 
in exercise (2) above). 


At this point, we have to face two natural objections: 


(1) The minor objection: our treatment of periodic signals is too simplistic. 
The restriction to trigonometric polynomials simulates idyllic situations. 
What about arbitrary periodic signals? 

(2) The major objection: the everyday signal is not periodic (although often 
locally periodic, particularly in acoustics, for example). How can we sample 
and reconstruct in these cases? 
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The answer to (1) is intimately linked to the following observation. 


Remark (Uniform trigonometrical approximation — the theorem of Fejér): 
Let f(0) be a continuous and P-periodic function (we shall put, as always, 
P=2W and T = x7). 
Consider the sequence of the arithmetical means of the Fourier polynomials 
Onf(@) associated with f(@) 


N-1 


pf 0) =~ Do onf(@), N21, 


n=0 


1.e. 


bef (9) = $ (oof (8) + orf (9) = See? + cg + Fee? 
usf(0) = 3(cof (8) + o1f (0) + o2f(0)) = xe-2e 477? + Zc_ye P79 


2 i 1 i 
+co + 2c,¢e?mi70 + zone Te, 


k het 
pv f(@)= (1 HE) ege?ia%, 
—N+1<k<N-1 


(One weakens the influence of the high frequencies.) 

Then the sequence (uy f(@))>1 converges uniformly to f(@): 

For every € > 0 there exists N > 1 such that |f(@) — un f(9)| < € for every 
OER. (f(0) and ua f (0) take the same values, up to an e-deviation). 

In this sense, every continuous periodic function “is” a trigonometric poly- 
nomial. 


Exercise 
Let f(@) be the continuous 27-periodic function 


0 forO <0 <7, 
Ce ee eee 


(a sawtooth curve). 


4 6 f ” 
Recall of (0) = : == cose i et | = . zl biseot 


Find N > 1 such that |f(0) — un f(9)| < 4 for every 0 € R. 


The answer to the objection (2) is given by the usual Whittaker-Shannon 
theorem (in an essentially non-periodic spirit). 
Let us begin with the mathematical basement. 
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3.3.3 The (Continuous) Fourier Transform: A Sketch 


Let f(t) = x(t) + iy(t) be a complex-valued function of the real variable t 
(“the time”). 
Suppose first that f = f(t) € L1(R) (“f = f(t) is summable over R”), ice. 
that the (Lebesgue) integral fess | f(t)|dé (is defined and) converges. 
Associate with f(t) the function of the variable w (“the normalized fre- 
quency” ) 


A +00 . 
fle) =f fleerae 
More explicitly: 
+00 


fw) = if (x(t) cos wt + y(t) sin wt)dt + if (y(t) coswt — x(t) sinwt)dt. 


—co —Co 


f(w) = F[f(t)] is called the Fourier transform of f(t). 


First Observation 


f (w) is a continuous function of the variable w (provided f(t) is a summable 
function). 


Basic example 


1 
AC) ve 1 for |¢| < oe 
0 for |t| > 3 
(this is a function which defines a “window of length 1”, according to common 
language in signal theory). 
~ 1 
Its Fourier transform: é!°l(w) = 2="°2" 


= sinc (#). This introduces defi- 
nitely the convolutional sine® 


: 1 forx = 0, 
sin c(z) = sin 72x for x A 0 


— we already became (implicitly) accustomed to it. 

We observe: é!°l(w) is infinitely differentiable, and each of its derivatives 
is bounded. On the other hand, é!°!(w) is not summable: 

je |él°l(w)|dw is divergent (although se éll(w)dw = 27). 


Before going further, we should briefly speak about the frequency variable. 


Remark A priori, the variable w describes a formal change of viewpoint 
(which proves to be extremely fruitful), but which should not be overloaded 
by obstinate conceptual interpretations.’ Down-to-earth frequencies — in the 


° We try to give a meaning to the fourth letter “c” — referring to the fundamental 
impulse response property of the function. 
” Do not forget that frequency without repetition is non-sense. 
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physical sense of the term — appear abundantly in the time domain descrip- 
tion of periodic phenomena, and are traditionally designated by the variable 
v. These (physical) frequencies will propagate into the (formal) frequency 
domain according to the identity: 


w= 2nV. 


But there is an important nuance in this seemingly flat change of variable: 
the variable v “hashes” the time, the variable w “forgets about” the time. 


Let us go back to our time—window example, which can be easily generalized. 
Consider 


e2mivot for |t| < 5, 
ell (t) = 
0 for |t| > 5. 


Then élol(w) = sine (45), wo = 27. 
Observe that we have, in general: if f(w) is the Fourier transform of f(t), 


then f(w — wo) is the Fourier transform of ciwot f(t),| 


We note: the function é!”°](w) = sin c (45°) is the characteristic function of 


2rivot “seen through the window 


the elementary periodic function el”l(t) = e 
of length 1, symmetric to the origin”. 

It attains its maximum for w = wo, corresponding to the fixed charac- 
teristic frequency, and tends symmetrically (with respect to the vertical line 


w = wp) to 0, for w — +oo, in smaller and smaller becoming undulations.® 


Exercise 


Show the following elementary result. 

Let f(w) = F[f(#)] and let a € R*. 

Then Flf(at)] = rh f(2). 
Consequence elvol (w) = n-sinc(n- *>**) (the index n indicates the length 
of the time domain window), i.e. if we “open” the truncation window for 
f(t) = e?""”t indefinitely, the corresponding characteristic functions will tend 
to the “function” 


spl _ [ c for w= wo, 
exo (w) = . otherwise 


(in a more advanced formalism, el (W) is simply the Dirac 6,.,,)). 


8 A wavelet — we shall see more of this specimen in the last chapter of this book. 
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In a first approach, this example will justify the denomination of w as the 
frequency variable. (We note, in the foregoing example, the correct appearance 
of the (time domain) frequency value vp in our w — formalism: passing to the 
limit creates a “point density” which indicates precisely this frequency — in 
normalized notation.) 


Remark Although the (continuous) Fourier transform makes the transit to 
the “frequency domain”, it is structurally hostile to periodicity: a non-zero 
periodic and continuous function f(t) is not summable over all of R, hence its 
(continuous) Fourier transform does not exist — at least as a function of the 
variable w. 

But if we extend the theory of the (continuous) Fourier transform to the 
tempered distributions, then we are in the correct situation: 


F YS Garret = S- Cup O(2nv0)- 


—m<Vvo<m —M<Vy<m 


The density defined by a trigonometric polynomial f(t) will be transformed 
into the characteristic distribution of the frequencies “occurring in f(t)” — 
every O(2nv)) is a Dirac distribution (i.e. a point density “centred in wo = 
2719” ). 

Let us now move on to the inverse Fourier transform. We define, for a 
function G(w) € L'(R), ie. a summable function of the variable w, 


1 pe 


g(t) = F[G(w)| G(w)el* dw. 


— on i 
We obtain thus a continuous function of the variable t. : 
For a continuous and summable function f(t), such that f(w) = F[f()] 
is (continuous and) summable, we have 


FO) =FFS(O), fw) = FFF), 


i.e. F and F are inverse transformations, respectively. 


Attention If fi(¢) = f(t) almost everywhere (the two functions are dis- 
tinct only on a set of measure zero — for example on a countable set), then 
fi(w) = f(w) for every w € R (the integration “does not remark” the change 
of values of a summable function on a set of measure zero). It is only possible 
to recover the initial function, by inverse Fourier transform, up to a (a priori 
unknown) set of measure zero. But every class of summable functions (equiv- 
alence = equality almost everywhere) admits at most one single continuous 
representative. In this sense, our reciprocity statement needs the coupling of 
summabilty and of continuity. 


Finally, let us state the Fundamental property of the Fourier transform. 
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The Convolution Theorem 


Define, for f = f(t), h = h(t) € L1(R), their convolution product g = f*hé€ 
L'(R) by 


+00 +00 
g(t) = / f(t — @)h(0)dd = a f(O)A(t — 0)d0. 


Then, F[9(¢)] = F[f()] - FlA()], ie. 9) = fw) - h(w). 
This is the characteristic and essential property of the Fourier trans- 
form. 
Towards a Hilbert Space Formalism 
Notation Iz, is the characteristic function of the subset EF € R. 
Example Vj_1/2,1/2)(t) = ell(t), 


Recall The Fourier transform of the function el”°](t) = e?7ivot ‘T_-1/2,1/9] is 
W—-WO 


the convolutional sine él”ol(w) = sinc (45“2), wo = 2m. 


Although the sinc function is not summable (over R), let us show altogether 
that 


Loge 
elvel(t) = = | alvol (w)e*dw. 
20 


—oo 


This will be an immediate consequence of the following exercise. 


Exercise (Fourier transform of the convolutional sine) 
Let ho(t) = sinc(t) = “2% (with continuous prolongation for t = 0). 


jh < 
Show that ho(w) = jx.) = ; al <n, 


(Help: use fe sin c(at)dt = Tal fora €R.) 
Solution 


m +00 , +00 
ho(w) = / sinc(t)-e'@'dt = i) sin c(t) - cos(wt)dt 


—Co —Co 


1 ia Ee ONE eas a dt. 


Bees at at 
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dt 
2 at at 


me! Pree 5, 1—2vy 
2 | [b+ ay) ~ jb— 2p] |" 


F 1 es pero Ee sin m(1 — 2v)t 


—co 


? i 
Whence: ho(w) ere Vl < 3, fe lila, 


O for |v} >5, | 0 for Jw| > 7. 


Consequence? 


Put h(t) = ho(t—k) = sinc(t—k), k € Z. This is a family of convolutional 
sines, centred at the particular k € Z. The family of their Fourier transforms 
(he(W)) rez are the sinusoids (in complex notation) 

hy(w) =e wk pliner keZ. 


We have already seen (frequency domain formalism of Fourier series) that this 
family is an orthonormal basis of the Hilbert space L?[—z, 7]. 


Question What is the characteristic Hilbert space property of the family 
(k(t) kez? 


The answer will be given by the theorem of Whittaker-Shannon. 


Consider the (Hilbert) space L?(R) of (classes of) square integrable func- 
tions on R. Our family of convolutional sines (hx)pez yields an example of 
typical elements of this space. 

Our first goal: to define an extension of the Fourier transform 


F : L?(R) —> L?(R), 
which will be an isomorphism of Hilbert spaces. 


Principal lemma. Let f,h € L(R)ML?(R). Then 


+00 = 1 +oo " = 
i f@h(t)dt = — f(w)h(w)dw (Parseval). 
In particular: 


/ i f(t)[?dt = ~ ‘y i f(w)|?dw (Plancherel). 


—oCo 


Proof Exercise (you have to use the Convolution Theorem relative to g = f «h, 
with h(t) = h(-t)). 


° We shall become convinced that the coupling: frequency domain window © time 
domain sinc function offers a robust skeleton to Fourier analysis. 
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Extension of F to the L?(R) by density 


How to define f(w) = F[f(t)] in case f = f(t) is square integrable, but not 
necessarily summable? 

L1(R) 9 L?(R) is dense in L?(R) (for the Hilbert space norm). Conse- 
quently, there exists (fn)nen in L'(R) 1 L?(R) such that ||/f — f,|| —> 0 
in L?(R). (fn)men is a Cauchy sequence in L2(R), hence (fn)nen is also a 
Cauchy sequence in L?(R), by the Plancherel formula. Let then f € L?(R) be 
the limit. We shall put F[f(t)] = f(w). 


Consequence F : L?(R) — L?(R) is an isomorphism of Hilbert spaces. 
In particular, we have the identities 


(f,9) = (f,9) (Parseval), 


\,fll = [l/l (Plancherel). 


Thus we can state: 
Our system (hz(t))ckez of shifted convolutional sines is an orthonormal 
system in L?(R). 


Remark The foregoing result is a remedy to a certain uneasiness in face of 
the L! — formalism: although we got reciprocity of F and of F on certain 
classes of continuous and summable functions, we were not able to speak of a 
bijection of the (time domain) L1(R) with the (frequency domain) L1(R): the 
Fourier transform of a summable function which is not continuous (modulo 
equivalence) cannot be summable (in this sense, the example of a “window 
function” is absolutely typical). 

Let us insist upon the structural particularity of the foregoing L? — iso- 
morphism. 

The L? Fourier transform, as well as its inverse, does not produce func- 
tions, but only classes of functions (i.e. distributions). 


Every L? — result which has been established via 


In simpler words: : ’ ; 
se Fourier transform, is only valid “almost everywhere”. 


3.3.4 The Sampling Theorem 


The Nyquist-Shannon theorem, in its popular version, reads like this: 

let f(t) be a signal such that f(v) =0 for |v| > W. 

Put T= sw: 

Carry out an equidistant (infinite) sampling for th, =k-T, k € Z. 

Then f(t) can be reconstructed by means of the samples f, = f(kT), k € Z. 

This statement offers a kind of universal key for the transit between the 
analog world and the digital world. But at first glance there are already 
some question marks: the common sense is (quite rightly) contradicted by the 
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mathematically indispensable appearance of an infinite sampling. The 
theoretical spirit asks: how can the function f(t), apparently rather arbitrary, 
be entirely determined by the sequence of values fy, = f(kT), k € Z? 

It is the second problem which will be our main concern. 

We shall give an L? — version of the Whittaker-Shannon theorem (the label 
“Nyquist—Shannon” has become universal tradition, so we shall not completely 
abandon it). First we look at a normalized situation. 


Consider the space L?[—7, 7] as a Hilbert subspace of L?(R): 
L?[-n,n] = {f €L°(R): fw) =0 for |u| > 7}. 
Let L?(R) be the time domain copy of L?[—7, z]: 
Li(R) = {f € L?(R): fe L?[-7,7]}. 


We obtain an induced isomorphism of Hilbert spaces 


F: 38) — Lm, 7, 


which makes correspond (hx (t))zez to (e7”*)pez.1° 
Consequence Whittaker—Shannon Theorem — Austere Version 
The system (hz(t))xez is an orthonormal basis of L7(R). 


More explicitly: consider, for f € L7(IR), the Fourier series of its Fourier 


transform!!: 


fw) =o flee" 
keZ 
with : . 
fk] =(fw),e*), keZ. 


Attention The convergence is a convergence in quadratic mean (i.e. with 
respect to the Hilbert norm) and the equality is an equality almost everywhere. 
All these natural operations will give in the time domain: 


f() = Venez f[khx(t) with f[k] = (f(t), he(t)), k € Z. 


Let us point out: the “coordinates” of f = f(t) with respect to the ortho- 
normal basis (h;(t))kez of L?(R) are the Fourier coefficients of f(w). 

But the representation of the f[k] as time domain inner products permits 
their identification as the values of a certain function: 


+oo 


(f(t), he(t)) = i f(t)ho(t — k)dt = (f «hg )(R) 


—Co 


10 We do not change notation for restricted sinusoids with compact support [—7, 7]. 

" Te. we do a 2z-periodic prolongation of the non-trivial branch of the Fourier 
transform, then develop it into a Fourier series, finally restrict everything back to 
]-— 17,7]. 
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with ho (t) = ho(—t). But, (f * ho )(t) = F[f(w)ho(—w)] and f(w)ho(—w) = 
(f- T-n,n})(w) = f(w). Hence, for f € L}(R) we have: (f * ha )(t) = Fif(w)]. 
) 


f() => filk)ha(t). 


kEZ 
We note: fi(t) is infinitely differentiable in virtue of the following result: 


Lemma Let G(w) be a function of the variable w such that each of the func- 
tions w™G(w), m > 0, is summable. 

Then g(t) = F[GWw)] = # en G(w)e'dw is an infinitely differentiable 
function (the differentiation is simply done under the integral sign). 


This holds manifestly for f,(t) (since f(w) admits a bounded support). 


Whittaker-Shannon Theorem — Normalized Version. 

Let f(t) be a square integrable function on R and such that f(w) = 0 for 
|w| > 7. 

Then the function f(t) = F[f(w)] is infinitely differentiable, and 


f(t) => filk)ha(t) 


kEZ 
(Attention: convergence and equality are in L7(R)). 


Commentary As stated above, the Whittaker-Shannon theorem is not a 
real interpolation theorem. The sequence (f1(k))ckez = (f[k]) rez is the se- 
quence of the Fourier coefficients of f(w), hence it does not depend on the 
values of f(t) on Z. 

On the other hand, the Shannon interpolator of (the series of convolutional 
sines) is the function f,(t), which differs from f(t) only on a set of measure 
zero. In this sense, we obtain a “generic” reconstruction of the line of f(t) by 
regularization (smoothing). 

If f(t) is smooth, i.e. infinitely differentiable, then f(t) = fi(t), and f(t) 
is effectively determined by the sequence (f(k))xez. 

So, do not forget: 


The Whittaker-Shannon theorem controls only smooth phenomena. 


This is by no means surprising, since an equidistant sampling in itself is 
a rather weak constraint. You have to create a rigid situation by a finite- 
ness condition (concerning the spectrum) and the (ensuing) regularity of the 
interpolator. 
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Exercise 


The orthogonal projection L?(R) —> L7(R). 
Let ho(t) = sin c(t) with ho(w) = I|_7,x)- 
Show: 


(a) For every f € L?(R) we have: f; = f * ho € L?(R). 
(b) If — fill = min{||f — gll : 9 € L7(R)}. 
Note: ho is the impulse response of the linear and time-invariant filter which 


associates with every (square integrable) signal f the nearest signal (for the 
Hilbert space distance) which has its spectrum supported in [—7, 7]. 


Remark We can also read the Whittaker-Shannon theorem as a true interpo- 
lation theorem — in direction digital —- analog exclusively. 

Let then (f[k])xez be an arbitrary time series, which is square summable: 
Ynez [FA]? < co. Consider the Whittaker-Shannon interpolator 


FQ) =S0 flalha(t). 
kEZ 
Then f(t) is infinitely differentiable and square integrable, and we get for 
its Fourier transform: 


f(w) =0 for |w| >7 
and 
f(w) = S- flklei#* for -tm<w<m. 
keZ 

This observation leads to the definition of the Fourier transform of a digital 
(square summable) signal — which will always be tacitly considered as the 
sequence of the samples of its Whittaker-Shannon interpolator. 

In order to finally arrive at the correct statement of the popular form of 
our theorem (recall the beginning of this paragraph), we only have to carry 
out the evident changes of variable. 


Nyquist—Shannon Theorem w-Version 
Let f = f(t) be a function which is square integrable on R. 
If f(w) = 0 outside [-, 4], 
then we may suppose f = f(t) infinitely differentiable, and we obtain 


FQ) => 0 f(&-T)hr(t—k-T) 


keZ 
with hr(t) = sinc(4). 
Nyquist—Shannon Theorem v-Version 


Let f = f(t) be a function which is square integrable on R. 
Assume f(v) = 0 outside [-W, W], and put T= x7. 
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Then we can suppose f = f(t) infinitely differentiable, and obtain 


FQ) =>) FE: Tart —k-T) 


keZ 
with h(t) = sine(4). 


Whittaker-—Shannon » 


Remarks How should we read the theorem of : 
Nyquist-Shannon 


(1) Let us begin with the mathematically correct situation: the signal f(t) 


is well known in its analytical form, hence f(w) is (hopefully) calculable, 


and one is able to verify that the hypotheses of the Nyquist-Shannon the- 
orem are satisfied. In this case, the reconstruction of f(t) by the Nyquist— 
Shannon formula may seem ridiculous, since f(t) is known (recall the 


development of cos 27t in a series of convolutional sines). 


(2) Pass now to a more realistic situation, which is mathematically a little bit 
frustrating: the signal f(t) is not explicitly known in its analytical form. 
The hypotheses of the interpolation theorem cannot be verified. The theo- 
rem will be applied by virtue of an extra-mathematical argument based on 
the intuitive sense of the notion of “frequency” (all acoustic phenomena). 


Then f(t) is nothing but the sequence of its samples. “Reconstruction” 


will simply signify “interpolation”: the Nyquist-Shannon interpolator will 
only give a smooth interpretation of the sequence of samples of f(t). 
Thus one creates a posteriori an analog signal which satisfies the hypothe- 
ses of the interpolation theorem. This answers another objection of math- 
ematical pedantry: a function with compact support can never satisfy the 
hypotheses of the Nyquist-Shannon theorem (see exercise (3) at the end). 
A finite sampling (and there are no other samplings in practice), which 
corresponds conceptually to the profile of a signal with compact support, 


will nevertheless give rise to a “mathematically sound” interpolator. 


(3) Let us come back to the validity of the periodic Nyquist-Shannon 
theorem. We have seen that there is equivalence with trigonometric in- 
terpolation via Discrete Fourier Transform. This suggests a strategy of 
primitive, but logical reconstruction: consider our (acoustic) signal f(t) 
as a trigonometric polynomial, with variable but locally constant coefh- 
cients cy = cz(t). Then the periodic Nyquist-Shannon theorem may be 
(locally) applied. One will sample to the frequency imposed by the phys- 
ical data, and one will shoulder off Nyquist—Shannon interpolation while 
practicing reconstruction via Discrete Fourier Transform. You will obtain 
this way a “piecewise trigonometric” adjustment (similar to the piecewise 


linear construction of curves in primitive computer graphics). 
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Exercises 


(1) We know that |sint| > 3 for km — 3 <t<kn—ZandkeZ. 
22 
(a) Deduce that fo" “@4at > 297k, Log #3, 
6 


1 


(b) Write Log 7-4 = Log (1+2- 1 ): 


5 
uo 6 


Observe that Log (1 2 ; 1.) >t: ois for almost all y > 1. 
6 


Y—6 


Conclude: the integral fea | S™4|d¢ is divergent. 


(2) Let f(t) =7- (sae) 


(a) Compute its Fourier transform f(w), and show that the hypotheses of 
the Nyquist-Shannon theorem are satisfied. 
(b) Represent f(t) by f(t), its Nyquist-Shannon interpolator. 


(3) Show that the (non-periodic) Nyquist-Shannon theorem can never rigor- 
ously apply to a concrete situation. 


Let f(t) be a signal, observed during a finite time (0 < t < L), and consider 
it as non-periodic: logically, you have to put f(t) =0 fort <Oandt> L. 

Show that in this situation the principal hypothesis of the Nyquist-— 
Shannon theorem can never be satisfied: f(t) and f(v) cannot admit simul- 
taneously a support of finite length. 


Help: suppose f(v) = 0 for |v| > W. Put T = sw: Then you get: f(t) = 
fit) = Cocnen fx S27 G ce With fy = f(T) and NT > L. 
f(t) must necessarily be infinitely differentiable on R. But f(t) = 0 for 
t<0. 
Hence: f(”) (0) = 0 for m > 0. 


Deduce that fo = fi =--- = fn =0. 


A 


Error Control Codes 


The first chapter (on lossless data compression) has turned around the ques- 
tion: How can we eliminate redundancies in the treatment of information? 
Now, we shall be concerned with the opposite question: Why should we feel 
obliged to introduce redundancies in the treatment of information, and how 
can we algorithmically control the situation? So, we have to speak about error 
correcting codes. 

The elementary theory of the subject has been treated extensively in the 
literature. Consequently, we shall focus our attention on two more advanced 
topics, which are best understood via ideas and methods of signal theory: 
The Reed-Solomon codes and the convolutional codes. In the first case, the 
decoding algorithm allows a pretty appearance of the Discrete Fourier Trans- 
form in binary polynomial arithmetic, in the second case, the mostly ac- 
claimed decoding algorithm (the Viterbi algorithm) is a kind of step-by-step 
deconvolution — in the spirit of digital filtering. 


4.1 The Reed—Solomon Codes 


In this first section, we shall show how the use of the Discrete Fourier Trans- 
form, with coefficients in certain finite fields of characteristic 2 (1 + 1 = 0)), 
makes the algorithmic control of a (very important) class of error correcting 
codes considerably easier: We shall speak about the Reed-Solomon codes. 


4.1.1 Preliminaries: Polynomial Codes 


Recall the fundamental facts, beginning with a simple Hamming code. Sup- 
pose that our information is encoded in the 16 binary words of length 4: 


0000,0001,..., 1111. 
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A transmission error of the information bitstream cannot be detected (and, 
a fortiori, cannot be corrected), since it necessarily changes meaningful words 
into meaningful words. 

In order to detect (and to correct) transmission errors, we have to introduce 
redundancies, i.e. we have to turn to an encoding where every code word 
(every significant word) has a “neighbourhood” of non-significant words — the 
appearance of which will reveal the transmission error (and, hopefully, the 
kind of the error). Let us encode our 16 binary words (of length 4) into 16 
code words (of length 7) in the following way: 


0000 a 0000000 1000 — 1011000, 
0001 an 0001011 1001 > 1010011, 
0010 ae 0010110 1010 es 1001110, 
0011 0011101 1011 — 1000101, 
0100 — 0101100 1100 a 1110100, 
0101 — 0100111 1101 es 1111111, 
0110 — 0111010 1110 > 1100010, 
O111 0110001 1111 > 1101001, 


We note: There are 128 binary words of length 7. Only 16 among them are 
elements of our code C (i.e. encode some information, have a meaning). 

We observe further: The minimum distance between the words of the code 
C is equal to 3 (the distance between two words is the number of positions 
where they are distinct): One cannot pass from a code word to another code 
word without changing at least three positions. 


Consequence Double errors (per block of length 7) can be detected. Single 
errors (per block of length 7) can be corrected. 


The reason: Every binary word of length 7 has a distance at most equal 
to 1 to some unique code word. 

The argument is the following: Consider the 16 “balls” of binary words 
of length 7 centered each at one of the 16 code words and filled with the 
7 derived words that are obtained if we change one of the 7 positions of 
the “central” code word. These 16 “balls” are mutually disjoint (since the 
minimum distance between the code words is equal to 3), hence their union 
has 16-8 = 128 elements, which is exhaustive. 

How did we succeed in creating this pretty situation? 

The answer comes from an arithmetical trick. Let us write down our in- 
formation words in |polynomial notation}: 

0101 = T? +1, 

1100=7T°+T?, etc. 


Now choose a generator polynomial g(T) for our code: 


g(T)=T3+TH+1. 
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The code words — in polynomial notation — are obtained via multiplication 
of the information polynomials with the generator polynomial: 


c(T)=i(T)g(T) 


Let us verify: 

We encoded: 1001 +— 1010011 

But, indeed: (T? + 1)(T?+T+1)=78+74+T+1. 

The algorithm for the detection of transmission errors is simple: Fuclidian 
division by g(T): 

One divides the transmitted word — in polynomial notation — by our gener- 
ator polynomial. If the remainder of division is zero, then one will suppose an 
error free transmission (we note that we recover at the same time the informa- 
tion word). If the remainder of division — the syndrome s(T) of the transmitted 
word — is non-zero, then one will suppose a single-error transmission. 

How to determine the error pattern, i.e. the position of the bit in error? 

Let c(T) be the correct (polynomial) word. A single transmission error 
at the position k (enumeration of the positions: 6.5.4.3.2.1.0) will create the 
received (polynomial) word 

vo(T) =c(T) + T*. 

Hence: s(T) = v(T) mod g(T) = T* mod g(T). This permits to deter- 
mine k. 


Example v = 1110011 ie. v(T) = T° +7°+7T4+TH+1. 
We have, in Fg = F.[T]/(T7? + T + 1): 


T =T+1, 
T =T'4T, 
TP SP oP A, 
T =T' +1, 


Hence: T6+7°+744+741 = (T?+1)+(T?7+T+1)+(17+T)+T+1l= 
T?+T+lie. s(T) =T7+7T+1=T° mod (T? +7 +1). We conclude: The 
error is at the position 5. We correct: (T) > ¢(T) = 7° +74+T +1. 
We recover the information: T° + T*++T+4+1: (T?+T+1) =T?4+1ie. 
i(T) =T3 +1 = 1001. 


Summary A polynomial code (of binary words of length n) admits the fol- 
lowing description: 


(1) The encoder. One multiplies the (polynomial) information words i(T) of 
length n—r (degi(T) < n-—1r—1) by a generator polynomial g(T) (of 
degree r): 
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(TL) = i(T)9(T). 
(2) The decoder. One divides the received (polynomial) word v(T) by the gen- 
erator polynomial: 


oT) = m(T)g(T) + s(T) 
3(T), the remainder of division, is the syndrome of v(T). 
If s(T) = 0, one supposes an error free transmission, and one recovers the 
information: 
i(T) = m(T). 
If s(T) £0, one has detected a transmission error. 


The correction of the error needs a deeper algorithmic control of F2[T]/(g(T)): 
you have to decide if s(T) is an error monomial, or binomial, or trinomial, 
etc., indicators of the current error pattern. 


Attention 


Error correction means: Find the nearest code word to the received word. 
One corrects always the most innocent error . 


Exercises 


(1) Let C be the Hamming code treated in the foregoing example (with gen- 

erator polynomial g(T) = 7? + T +1). 

Decode the following block of four words: vyvov3v4 = 1110111.011001. 

1010101.1110011. 

Let us go a little bit further: Consider the Hamming code C with generator 

polynomial g(T) = T4++T7 +1. 

The information words will have length 11 (represented by the binary 

polynomials of degree < 10), the code words will thus have 15 bits each. 

(a) Show that the 15 non-zero syndromes correspond precisely to the 15 
single-error patterns (recall the cyclic group (F2[T]/(T* + T + 1))*). 

(b) Deduce that every binary word of length 15 is either a code word or 
at distance 1 of a unique code word. The minimum distance between 
the code words is thus 3. 

(c) Decode v, = 111011101110111, then ve = 100101010001010. 

Let C be a binary polynomial code, with generator polynomial g(T), and 

let n be the length of the code words. Show: C is cyclic (invariant under 

cyclic permutation of the bit-positions of its words) == 4g(T) divides 

T’ +1. 

[This is true for the two Hamming codes that we have treated before: 

T? +7 + 1 divides T* +1, and T* + T+ 1 divides T™ + 1. 

Help: Let o be the cyclic-left-shift operator; then ov(T) = Tv(T) mod 

(T" +1). ] 


— 
i) 
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(4) (Towards the Meggitt decoder). Let C be a (binary) cyclic code, g(T) its 


Na 


NH 


generator polynomial, n the length of its words. v(T) a binary word of 
length n, in polynomial notation, s(T') its syndrome. v'(T) = ov(T) the 
cyclically left-shifted word, s’(T) its syndrome. Show that 


s'(T) = Ts(T) mod g(T) 


Let C be the cyclic binary code defined by the generator polynomial g(T’) = 
TS+7°+ 784+ T*415 (144741014729 +7? +T +1) (dividing 
T!° +1). The length of the code words will then be equal to 15, and the 
length of the information words will be equal to 7. 
Accept. The minimum distance between the code words is equal to 5. C will 
thus admit the correction of single or double transmission errors (per block 
of 15 bits). Decode v; = 110101011001000 and v2 = 001011100001110. 
[Help: The syndromes s9(T’) = 1 and sz.o(T) = T* + 1,1 <k < 7, corre- 
spond to error patterns of a single error at the last position, and of double 
errors, concerning the last position and the kth position before. Using 
the result of exercise (4), you can reduce — via iterated cyclic shift — the 
decoding to the identification of one of these standard syndromes, while 
counting the number of necessary cyclic shifts).] 

The binary Golay code The information words: The 4,096 binary words of 

length 12. The generator polynomial: g(T) = T''+7'°4+ 784+ 7°+T* + 

T? +1. The code words (in polynomial version) c(T) = i(T)g(T) are then 

of length 23. 

Cyclicity. T?3 +1 = (T +1)9(T)9(T), with 9(T) =TU+T9+T7 +78 + 

T?+T+1. 

Accept. The minimum distance between the code words is 7. 

Thus C admits the correction of error patterns of a single, a double or a 

triple error (per block of 23 bits). 

(a) Let v be a binary word of length 23, derived from a word of C after 
a transmission error of three faults at positions 18, 13 and 2. Which 
will be its syndrome? 

(b) You receive the word v = 10111101100101111010011. Decode and re- 
cover the information. 


4.1.2 Reed—Solomon Codes 


The Reed-Solomon codes are polynomial codes, but on a (slightly) superior 
level to that of our foregoing description. 


The information units — the letters — are no longer the bits 0 and 1, but, 


for given n > 2, the 2” binary words of length n. In other words, we will 
have two different levels of polynomial arithmetic: First, the arithmetic of 
the finite fields Fon — i.e. that of the remainders of division by an irreducible 
binary polynomial of degree n — which will be the arithmetic of the letters of 
the chosen alphabet. 
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Then, the arithmetic of the polynomials with coefficients in the chosen 
field Fon, which places us in the algorithmic context of polynomial coding. 
Let us fix the notation (and the arithmetic) for the sequel: 


Fy 


I 


Z/2Z = {0,1}, 


F, = Fo[2]/(2? +241), 


Fg = F.[z]/(2° + 2+ 1), 


co +e¢+4+1), 


‘64 = Fo[z]/ 


( 
( 
‘16 = Fo[a]/(x* + 24+ 1), 
( 
( 


Fo56 = 'o[a]/ x batt a? +4741), 
Then, we have to indicate the parameters of our presentation: Let Fon, n > 2, 
be chosen, and let N be a divisor of 2” — 1. 

The Reed-Solomon code RS(N,N — 2t) over Fon will be a polynomial 
code, with generator polynomial g(T’) € Fan[T]. The degree of g(T): 2t. A 
code word will have N letters in Fon, hence it will be a block of N - n bits. 
An information word will have N — 2¢ letters in Fon, hence it will be a block 
of (N — 2t)-n bits. RS(N, N — 2t) guarantees the correction of error patterns 
of maximally ¢ errors. 


Attention One error means one letter in error, t.e. a block of n bits in 
error. 


An illustration: Consider the code RS(15,7) over Fas¢. 
The length of the code words: 15 8-bit bytes = 120 bits. 
The length of the information words: 7 bytes = 56 bits. 


RS(15, 7) admits the correction of 4-error patterns, i.e. of four bytes in error (in 
a 15-byte word). For example: A transmission error which affects 26 successive 
bits (inside a block of 32 bits) can be corrected. 


This points at the great practical interest of the Reed-Solomon codes (a 
convincing application: the error control code used in the CD system employs 
two concatenated Reed-Solomon codes, which are interleaved cross-wise: the 
— Cross-Interleaved Reed-Solomon Code (CIRC) — which is able to correct 
error bursts up to 3,500 bits (2.4mm in length) and compensates for error 
bursts up to 12,000 bits (8.5mm) that may be caused by minor scratches). 

The theory (and the practice) of the Reed—Solomon codes owes its elegance 
(and its efficiency) to the interaction of polynomial ideas with “spectral” tech- 
niques based on the use of the Discrete Fourier Transform over the finite fields 
Fon, n = yA 
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Definition and First Properties of Reed—Solomon Codes 


Fix n > 2, and consider Fan = Fo{[2]/(p(x)), where p(x) is an irreducible 
(primitive) binary polynomial of degree n. 

Let N be a divisor of 2" — 1, and let w = wy € Fn be a (primitive) Nth 
root of unity: wN = 1, but w* 41 for 1 < k < N. (We note that whenever 
p(x) is primitive, we have w = aN’ with N’ = aa 

Hence we dispose of the Discrete Fourier Transform of order N14 


Fn : (Fon)% —? (Fee). 


Recall our notational convention. The components of the transformed vec- 
tor of a vector with variable components will be noted in upper-case letters. 
Fix now t: 1<t<%. 


Definition of the Reed-Solomon code RS(N, N — 2t) over Fan: 


Zo Zn—2 = 0 
Za % Zn—2t+1 = 0 
RS(N, N —2t) = |e (Fn): 
ZN-1 Zn-1 =0 


RS(N, N — 2t) is the linear subspace of (Fan) which consists of the N- 
tuples with the property that the 2t last components of their Fourier transform 
are equal to zero. 


Example The code RS(5,3) over Fig = Fa[a]/(a* + a +1). 


w= Ws = 23 


Lie I PL TL 0001 0001 0001 0001 0001 
la? «® 2 2? 0001 1000 1100 1010 1111 
F5=] 12° 222 «9 | = |] 0001 1100 1111 1000 1010 
1a? 2° zl 7 0001 1010 1000 1111 1100 
Lala? 2% 2 0001 1111 1010 1100 1000 
ZO 
Z1 
_ 5, 43=0 
RS(5,3) = : € (Fie)? : Z,=0 
ZA 


' We only need a group of Nth roots of unity in (F2n)* — in formal analogy with 
the group of Nth roots of unity in C*. Note that the matrix Be is simply the 
“conjugate” of Fy: Replace every coefficient by its multiplicative inverse. Since 
N is odd, the factor 4 a 
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20 
_ e € (Fig)®: 20 + (1010)21 + (1000)z2 + (1111)z3 + (1100)24 = 0 
~ . 16) * zo + (1111)21 + (1010) 22 + (1100) 23 + (1000) 24 = 0 
3 
ZA 


This gives the code RS(5,3) over Fig by its parity-check equations. 


Exercise 


(1) What is the probability that a word of 20 bits (chosen randomly) is a code 
word of RS(5,3) over Fig ? 

(2) Still the Reed-Solomon code RS(5,3) over Fig. 
You receive the word 0001.0101. * * * *. ** * *.1010, where eight successive 
binary positions are illegible. Supposing an otherwise error free transmis- 
sion, decode and recover the information. 


(3) The code RS(7,3) over Fe4 = Fa[z]/(a® + x +1). Recall the cyclic group 
of 63th roots of unity in Feq: 
ce = a+ a> =2°>4+2, eta tei tet 
eg =274+a, 2% =e? +a4+1, 2 = 744+ e5 41, 
a? = 22 +22, ee — gt eta, 26 = 27> 42442, 
2 —2t+23, o28 — gt 4 93 4 2? et =e 422? 4e41, 
xO = 7d 4 2, 229 =o +244 23, f8 = 78 4 2 41, 
etiaeg+r4+1, eae +et+e4+1, e® =e7t +2342, 
at? = 2? 41, et Bag? 4 7, 229 = 2 4 24 +2, 
a =23 +2, a =o +1, etaeg +a34+a41, 
a4 = gt 4 2, 2 —2t+2, ao? — ot +e? 41, 
ed = 7d 4 23, 84 = 75 4 gy? 23 — 75 +23 4 x, 
a6 =e¢44+e41, a> =o +a +1, o4 et te? +a41, 
a =7° 40742, ee =e7t*+a7%74+2 2 =27>° 42° bar +a, 
a8 =73 +a74+24 1, oe? = 27> +25 +27, 2S =et* tae +a?+4+ 24 1, 
e%=e7t+a5407? +2, oe =e74+e5 +41, el =e 4+ 744+ e2 +2 +2 
22? = g® 4 gt 4 e3 4 2?, 2 — go tettertoa, 28 =e 4+ e744 a3 +4 

se 1; 

gta x? at z’+a+l, 79 =? a? | a x+1, 7°? = 2? | at x a? 1, 
eae tatte +1, at! = x4 a? | a 1, 7 —~ et etter +1 
c= 27> +25 1, cae +at+4 23 x; et = gi tatt 1, 
a ale a3 > at | a x+1, 78? = 2? + 1 
“The” seventh (primitive) root of unity : w = w7 = 2° 


(a) Write the matrix F7 of the Discrete Fourier Transform of order 7 over 
Fea. 

(b) Write the parity-check equations for RS(7,3) over Fe4 = Fa[a]/(x® + 
x+1). 
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1 
x 
x 
Fz = x 
x 
x 


PRR Re RB 


iy tae x 360 27 18 


000001 000001 000001 000001 000001 000001 000001 
000001 011000 001111 001110 010110 011001 010111 
000001 001111 010110 010111 011000 001110 011001 
= 000001 001110 010111 001111 011001 011000 010110 
000001 010110 011000 011001 001111 010111 001110 
000001 011001 001110 011000 010111 010110 001111 
000001 010111 011001 010110 001110 001111 011000 


RS(7, 3) = 23 : 45 27 9 54 p36 


Zo + 2°°% zy 2° zg + 2°" zg e'z4+u "25 +2 


(3) What is the probability that a word of 42 bits (chosen randomly) is a code 
word of RS(7,3) over Fea? 
(4) Some immediately recognizable code words. 
(a) Let z9.21.29.23.24 be a “constant” word of five times four bits: zo = 
21 = 22 = 23 = 24. Show that it is a code word of RS(5,3) over Fig. 
(b) Let 20.21.22.23.24.25.26 be a “constant” word of seven times six bits: 
Zo = 241 = 2 = 23 = 2% = 25 = 2. Show that it is a code word of 
RS(7,3) over Fea. 
(c) Generalize. 
(5) Cyclicity of the Reed-Solomon codes. 
(a) Let 29.21.22.23.24 be a code word of RS(5,3) over Fig. Show that then 
24.20-21-22.23 is also a code word of RS(5,3) over Fyg. 
(b) Let 20.21.29.23.24.25.26 be a code word of RS(7,3) over Fea. Show that 
then 26.20-21-22-23-24.25 is also a code word of RS(7,3) over Fea. 


We return to the general case: 
Important Observation 
Write for the vector (for the word) 
20 
21 
ZN-1 


the polynomial p(T) = 29 + 1T +++: +2y—-1T%7!. Then 
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20 p(1) 

Z1 p(w) 

Fn 22 = p(w”) 
eva} \pwX-}) 


We insist: The Fourier transform of a word of length N is obtained as the 
sample vector of the polynomial of this word, evaluated “around the discrete 
unit circle of order N” in Fon. 

Put now 

CT) SPS oF Sg OP Se) 

Then, RS(N,N — 2t) = {p(T) : g(T) divides p(T)}. The argument: 
RS(N, N — 2t) is the set of the words of length N which, in polynomial 
notation, take the value zero for the 2¢ “last” powers of w, i.e. for w¥~??, 
wN-2t+1 yy N=1. 


Se oe 


Consequence RS(N, N — 2t) is a polynomial code over Fo» with generator 
polynomial g(T) = (T — w-*)\(T — wN-2t4+1) |. (T — w-1) € Fon [T]. 


Exercises 


(1) Compute the generator polynomial g(T) for RS(5,3) over Fig. 
Verify that the code word (of length 5) which corresponds to the coeffi- 
cients of g(T) satisfies the parity-check equations. 

(2) Compute the generator polynomial g(T’) for RS(7,3) over 

Verify that the code word (of length 7) which corresponds to the coeffi- 

cients of g(T) satisfies the parity-check equations. 

Cyclicity of the Reed-Solomon codes. Consider the general case of a Reed— 

Solomon code RS(N, N — 2t) over Fon. 

(a) Show that the generator polynomial g(T) divides the polynomial 
TN +1. 

(b) Deduce the cyclicity of RS(N, N — 2t) : If 29.21....zN_2-2n-1 i8 a 
code word, then zy _1.29.21....ZN—2 is also a code word. 

(4) In the following list of words, how many are words of the code RS(5,3) 


(3 


mH 


over Fi¢? 

0000.0011.1110.0101.0111 0000.0110.1111.1010.1110 
1110.0000.0110.1111.1010 1110.0101.0111.0000.0011 
0111.0000.0011.1110.0101 1010.1110.0000.0110.1111 


0000.0011.1111.0101.0101 1000.0111.1011.1010.0000 
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(5) In the following list of words, how many are words of the code RS(5,3) 


over Fi? 
0001.1100.0101.1101.0101 1000.0110.0111.0110.1111 
0000.1100.0101.0001.0000 0010.1011.1010.1001.1010 
0111.0110.1111.1000.0110 0000.0000.0000.0000.0000 
1101.0101.0001.1100.1111 0000.0000.0001.0100.1010 

(6) In the following list of words of the code RS(7,3) over Fea , find the masked 
letters. 
29.21.0°9 94 1.739 1 29.0.22.0.24.0.26 
pS Bs paca eee ae a? py za. taceece’® 
x8 21 .1.20°5 24.25.26 By Fie te 5% 


The encoding algorithm is simple: 


Let a(T) = ap +ayT +--+ an—2t-1T%~7*-! be the polynomial of the 
information word 


E (yao 


QN-2t-1 


c(T) = a(T)9(T) = 2+ 2T+-:-+2n-1T%~! will be the polynomial of the 
code word 


associated with 


AN—2t-1 
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Exercises 


(1) Still our code RS(5, 3) over Fig. 
Verify that the information word 1010.1111.0101 is coded in 
0001.1100.0101.1101.0101. 

(2) Change your viewpoint: Consider the code RS(5,3) over Fig as being 
parametrized in the following way: 


20 Zo 
ZI Ly 
22 = Fe 22 
23 0 
ZA 0 


Find the information word ag.a1.a2 of the polynomial encoding which 
corresponds to Zp.Z1.Z2 = 0001.1010.1111 of the matrix encoding. 

(3) Consider now the code RS(7, 3) over Fea. 
Find the code word associated with the information word do.a,.a2 = 
010111.011001.000001. Recall: g(T) = «°° + 2°47 4+ T? + 2°73 + 7%. 


Decoding and Error Correction 


Remark RS(N, N — 2t) permits the correction of t-error patterns. 


Proof Let us first show that every non-zero word of RS(N, N — 2t) admits at 
least 2t + 1 non-zero components: 


Zo 
20 ; 

oa ZN—2t— 

Fy a i 1 
ZN-1 : 


Consider q(T) = 29+ Z:T +--+» + Zy_—at-1T%~7*—!. q(T) admits at most 
N — (2t +1) zeros. But 


. q(1) 
si q(w") 
Mel ares Ie 
No qu N*1) 


which yields the claim. 


Hence: Two distinct words of the code are distinct in at least 2t+ 1 po- 
sitions. (The argument: The word sum is a code word, and it is non-zero 
precisely at the positions where the two words are distinct). 
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As a consequence, a transmission error of (maximally) ¢ faults in a given 
code word will create a word in error which remains still at a distance > t+1 
from every other code word. 

Now let us pass on to the error-correcting algorithm. A received word v 
can be written as the sum of the correct word z and of the error word e (the 
components of e indicate the “error events” ): 


Vo 20 €0 
U1 CAN El 
= + 
UN-1 ZN-1 EN-1 


— | 4-2-1 | 4 | Ey—ae-1 
Vip 0 En-1 


Hence: Starting with the received word v, one knows the 2t last positions 
En—2t,.-.,En—1 of 


€0 
€1 
F'n ; 
E€N-1 
We have to compute the components Eo, ... Fy —2¢—1 in function of the 2t 


last components 
under the hypothesis that there are at most t non-zero components ej, 
O0<i<N-1. 


Lemma There exist Ay, A2,..., At with 


i Ep 0 
t Ey 0 €0 

fe , = oa €1 admits at most t 

0 non-zero components. 
; €N-1 

0 En_-1 0 
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€0 
Proof : admits at most t non-zero components <=> there exists 
E€N-1 
ao 
; , admitting at most t zero components, and such that ageg = 
QN-1 
aye, = +++ =ay—-1en-1 = 0. 
Write 
ao Bo BU): 
i ee = p(w) 
QN-1 Bn-1 p(w N+1) 
with p(T) = 6o + Ail +--+ + Bw-1TN7. 
Our hypothesis becomes: p(w~1) = 0 for at most t exponents J1, j2,..-,Js- 


Write p(T) = po(T)q(T) with po(T) = (T—w~h)(T ws). (Tw). 
q(w) £0 for 0 < 7 < N—1. We thus can replace p(T) by p(T’) and normalize 
to 1 the constant term of po(T) : po(T) = 1+ AiT + AeT? +--+ + A:T*. But 


1 
ri Eo po(1)eo 
: Ey po(w "er 
Fo\| ArJ«| oi |= 
0 . 
; ENn-1 po(w Nt" )en—1 


according to the Convolution Theorem for Fry ' This provides our claim. 


When applying the lemma to our situation, we obtain the following linear 
system: 


Eo = AyEn—1 + AsEn—-2 +++: + ArEn—+ 
Ey, = Ay Eo + AgEn—1 +++ + ApEn—t41 


Eny—t = Ay En—t-1 + AoE n—t-2 +++: + At ENn—2t 


En) = Ay En—24+ AgEn—3 +++: + ApENn—t-1 


This gives a t x t linear system for A;, Ao,..., At: 


En—t-1 En-t-2... En—2t Ay En-t 


Eny-2 En_—3 ... En—t-1 At En-1 
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One resolves, i.e. one finds A;, Ag,..., Ay, then one computes Eo, Fy,... 
E'n—t-1. 


d 


Let us sum up the decoding mechanism for our two favorite Reed-Solomon 
codes: 


RS(5,3) over Fig RS(7,3) over Fe4 
Correction of t = 1 letter in error Correction of t = 2 letters in error 
per transmitted word per transmitted word 
Bs V3 
Bs V3 Ex, |_| Va 
te (#2) = (¥8) ra [2] =[¥ 
Eg Ve 
Compute A, : Compute Aj, Ag: 

_ Ey E3 A,\ {Es 
So ee (ae) G  \ Ee 
then: then: 

Eo = Ay E4 Eo = A, Eg =r Ao Es 

Ey = A, Eo Ey = A, Eo + Ao E6 

FE = A, Ey Ey = A, Fy = Ag Eo 
Exercises 


(1) Consider the code RS(5, 3) over Fig. 
You receive the word 1000.0100.0111.0110.1111. 
(a) Decode (verify, whether there is a transmission error; in the affirmative 
case, correct the error). 
(b) Recover the 12 information bits. 


Solution (a) Apply first the Fourier transformation of order 5: 


Vo vo 11 1 a x £ 

Vi v1 12? 26 2° 2? x etertt+e+1 
Vo | =Fs] vo} =] 12° c? 2? 2 oO | =] eter tetl 
V3 v3 12° 2° x? 7 x e+aetl 
Vi V4 Lal? 2° 2 x gi? e+e +1 


Hence: E3 = 2? +2+1, Ey = 23 +2741. The single-error correcting 
algorithm starts with the equations 
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Ww 


WwW 


Na 


a 
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Eo = A, Ea, 
Ey, = A, Eo, 
Ey = Ai Fy, 
E3 = Ay Eo, 
Ey = Ay E3. 34,2 13 
We thus get: Ay = of == 73 
Eo x Zo 0 
Ey xa+1 Zy w+ x? 
Ey |=) 2+a+1 | hence} Z | = x? 
E3 x? +a+1 23 0 
Ey x + x? + 1 Z4 0) 
The code word: 
ZO 0 x 1000 
Zy x+ a? ue+en 0110 
zg | =F} x? = z+ae+1 =| 0111 
23 0 e+e 0110 
24 0 ee+a%*+atl 1111 
Divide (1111)T4+(0110)T?+(0111)T?+(0110)T+ (1000) by the generator 


polynomial T? + (0101)T + (1100): 
(xe?T4 4 2° TS + 2 T? + 2®T + 23) : (T? + 2®T +28) = 2? T? +e? 
ol? T4 4 oT? 4 oT? 
oT? +.2°T +23 
727? 4. oT + 8 
0 
The information word is: 1111.0000.1111. 


Once more, the Reed-Solomon code RS(5,3) over GF'(16) = Fa[a]/(a4* + 

x+1). 

You receive the word 

1000.1111.0111.0110.1111. 

(a) Decode (verify, whether there is a transmission error; in the affirmative 
case, correct the error). 

(b) Recover the 12 information bits. 

Consider now the Reed-Solomon code RS(7,3) over Fe4 = Fa[a]/(a° + 

el) 

You receive the word — 000000.000000.000000.000000.000111.101101. 

101010. 

(a) Decode (verify, whether there is a transmission error; in the affirmative 
case, correct the error). 

(b) Recover the 18 information bits. 

Still the Reed-Solomon code RS(7,3) over GF(64) = Fo[x]/(x® + « + 1). 

You receive the word 

100001.100110. x 2 2 x 2 2k, 2k 2k 2 OK KOK KKK KK. KK KK -*.111000, where 24 

successive bits have become illegible. We suppose an otherwise error free 
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transmission. Find the relevant code word and recover the 18 information 
bits. 

(5) Consider Fg = {0,1, 29, x18, x?”, 2°6, «#5, 254} C Fe4. How many words of 
the code RS(7, 3) over F¢4 have all (seven) letters in Fg? 

(6) Define the Discrete Fourier Transform of order 7 over Fg4 with respect 
to @ = x°* (we exchange the matrices F; and Ee Fp = re and 
F>' = F;). Which will be now the generator polynomial g(T) of the new 
Reed-Solomon code RS(7,3) over Fea ? F 

(7) How many words have the two Reed-Solomon codes RS(7, 3) and RS(7, 3) 
over F¢4 (cf the preceding question (6)) in common? 

(8) Consider the code RS(7,3) over Feq4 = F2[2]/(2® + x +1). 

Decode the word v = 000000.000000.000011.011101.111111.000100.111000, 
Le. 

— verify if there is a transmission error; 

— in the affirmative case, correct the error; 

— recover the 18 information bits. 

(9) The same situation. 

(a) Decode the word v = 010110.001111.001110.011001.011000.011001. 
010110, ice. 
— verify if there is a transmission error; 
— in the affirmative case, correct the error; 
— recover the 18 information bits. 
(b) Decode the word v’ = 011000.011001.010110.010110.001111.001110. 
011001. 
Try to make use of (a)! 
(10) Still the Reed-Solomon code RS(7,3) over Fea = Fo{x]/(a® + x + 1). 
You receive the word 
v = 010110.000001.000000.000000.010110.010110.000000. 
(a) Decode (verify if there is a transmission error; in the affirmative case, 
correct the error). 
(b) Recover the 18 information bits, by means of the polynomial method. 

(11) The situation of the preceding exercise. 

One receives the word 

v = 000000.000000.000001.000001.001110.000001.001110. 
(a) Decode. 

(b) Recover the 18 information bits. 

(12) The Reed-Solomon code RS(15, 7) over Fasg = Fa[x]/(a8+a4+a23+27+1). 
Let us first sum up the situation: The letters will be 8-bit bytes, identified 
with remainders of division modulo p(x) = 2° + 24 + 2? + 2? +1. The 
multiplicative group of the non-zero residues is cyclic, and generated by 
(the remainder of division) x: This gives the log table for the 255 non-zero 
bytes in the preliminary section to the cipher AES-Rijndael. 


Notation 11111111 = ff=2'? +25 +25 +e4+ 23 +2? +¢41= 2", 
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A code word will have 15 letters, hence 120 bits. We will be able to correct 

transmission errors of four bytes per code word. 

(a) First compute the generator polynomial g(T) of the code RS(15,7) 
over Fo56. 

(b) Decode the word 


V = Up -++Ui4 =ca 5d a2 48 ca 7d 7d 58 38 f4 80 15 48 20 ff ‘(hex 
notation) = 273 a6 209 7226 p73. 243.243. p41 p201 97230 7 p11 97226 95 


a a 
w® ie, 


— verify if there is a transmission error; 
— in the affirmative case, correct the error; 
— recover the 56 information bits, by the polynomial method. 


[Help: g(T) = a3 +1367 4 gl87 72 ¢ p21 73 4 gl87 P44 glOTS 4 723876 4 
85-7 8 
ool’ +T 


The matrix of the Discrete Fourier Transform of order 15 over Fo5¢: 
Ve 9 GS ee ot | aes ee TP cole, ols, al 
1 gi? 34 gt 8 85 102 gil? 186 id8 i790 87 294 2?) 7238 
1 g4 88 102 136 iO 204 238 gi? gt 85 git? 153 187 22h 
1 got 102 153 204 get 102 153 204 get gpl 04 158 204 
1 78 136 294 gi? 85 758 2?) 34 192 i790 7238 7?) gilts 87 
1 85 gi70 1 85 iO 1 85 i790 1 85 iO 1 85 gi70 
1 102 204 gt 153 gp 02 204 gt 153 102 204 got 153 
Fi5 = 1 gil? 238 192 2?) 85 7204 78 87 gt i790 34 7158 gi? 36 , 
1 186 gi? 153 34 iO get gi 87 88 204 85 221 gp 102 238 git9 
1 id3 gh 294 192 758 gh 294 192 758 gl 204 192 
s 
1 iO 7 1 i790 85 1 gitO 8 1 i790 785 1 gitO 78 
1 gl 87 git9 gt 7238 gi gp 102 34 22h 153 85 gi? 204 186 88 
1 204 id8 192 gl 7294 id3 192 gt 204 7158 102 gt 
: 
1 2?) 87 i538 wii9 785 yl gi? 238 294 i790 7186 102 78 34 
1 238 g22t 204 187 iO i583 186 git9 102 85 88 get 34 gi? 
then, 
38 
Vo Vo x 
73 
Vi Vi x 
Vo V2 160 
00 
V3 V3 x 
50 
V4 V4 x”? 
24 
Vs Vs At 
96 
Ve Ve x 
66 
V= V7 = Ey = x 
2 
Vg Eg x 8 
52 
Vo Eg xr 
89 
Vio Fro x 
80 
Vit Fy, x 
73 
Viz E12 x 
46 
Vis Fiz x 
58 
Via Fis x 


The linear system for Ai, A2, A3, As becomes: 
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89 wp? 182 166 Ai 180 
180 89 2 182 A> 3 
3 180 89 gp? As = 146 
46 3 180 89 Ag 58 


By Gauss-elimination, we obtain: A, = 1®8, Ay = «°!, Ag = x1°?, Ag = 2°. 
Ey = 2!33 Fy =a! By = 223 Bs =)” 
Thus: 
E, = 2° Ex = 21% Be = 312? 
The transmitted code word: x7?.2°6.a?09 7276 473 7243 7243 7°? 7? 0.0? x 
226 gp? ght 


The information word: apa ,a2a3a4a5a¢ = ff 00 ff 00 ff 00 ff = (1 4+ a7 + 


a a x°) 


4.2 Convolutional Codes 


The convolutional codes occupy a similar place in the domain of error correcting 
codes as arithmetical coding in the domain of data compaction: A bitstream, in 
principle unlimited, will be transformed into a binary code stream (in principle also 
unlimited) by means of a kind of arithmetical filtering which works in complete 
formal analogy to usual digital filtering — with the laws of binary arithmetic, of 
course (recall the cryptosystem AES and the transformation of the state array by 
circular convolution). 

The decoding algorithm has to solve a convolution equation which will be, in 
general, perturbed by a transmission error. If the number of errors per “inspection 
window” does not exceed a certain threshold, then the decoding algorithm (usually 
the algorithm of Viterbi) will properly produce the initial information stream and, 
in this way, will correct the transmission error. 

Our presentation avoids the usual geometrization of the theory of convolutional 
codes (on the one hand, as trellis codes, on the other hand, as finite automata). 
The reason: Most algorithmic problems about convolutional codes become rather 
inaccessible under the mask of “convincing” geometrical arguments. 


4.2.1 Encoding: Digital Filtering in Binary Arithmetic 
Generalities: 


A convolutional encoder is defined — as already mentioned — in formal analogy with 
a digital filter. It will transform an |information bitstream] into a |code bitstream|. 

The code bitstream has to be “thicker” than the information bitstream: There 
is no detection and no correction of transmission errors without redundancies! This 
means: More code bits than information bits per unit of time. More precisely: The 
information bitstream aga1a2---a¢--- is presented in frames aj, j > 0. 

Every frame is a binary word of (fixed) length k, k > 1. We shall adopt a notation 
in columns, in order to underline the parallelism of arrival: 
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a10 ai1 ait 
a20 a21 Gat 
ao = ‘ > a= pore At = “ ; 
ako Qk1 Akt 
The code bitstream cocic2---cz:-- is equally presented in frames c;, j > 0. 


Every frame is a binary word of (fixed) length n, n > 1: 


C10 Ci Cit 
C20 C21 C2t 

co = ’ C= ’ Le = ’ 
Cno Cn Cnt 


In order to guarantee that the code bitstream is thicker than the information 
bitstream, one demands n > k. We will speak of an (n,k) convolutional code. We 
shall only treat the case k = 1: Every information frame will be a single bit. The 
transformation of the information bitstream into the code bitstream will be carried 
out by an encoder which functions like (the convolution with) the impulse response 
of a digital filter. 

In order to obtain a neat formalism, let us introduce a variable x, which will be 
discrete-time carrier. Then x‘ signifies “at the moment t”. The information bitstream 
thus becomes a formal power series a(x) = )> 2S a,x’ where the information frame 


at “at the moment t” is the coefficient of x’. 

The code bitstream will also be a formal power series c(x) = > 
coefficients c; are binary n-tuples. 

A convolutional encoder thus becomes a transformation T : F&{[x]] © — 
FS ([r]] between two groups of formal power series with vector coefficients 
(of length k and n, respectively), where T will have a rather specific form: 


e(x) = T(a(x)) = G(x)a(x) 


cx’. The 


t>0 


G(a) is a (fixed) polynomial with matrix coefficients: G(x) = Go+ Gia +Gox" 4 
++ +Gmz™ with G; €FZ**[2], O<j<m. 


We insist: The coefficients of G(a) are matrices over Fz = {0,1}, consisting of 
n lines and k columns. They can be multiplied with the information frames, thus 
producing vectors in the format of the code frames. The variable x commutes with 
everything in order to guarantee that the result of the operation becomes an element 


of F3[[e]]- | 


Formally, G(x) occupies at the same time the place of the generator polynomial 
of a polynomial code and of the impulse response of a digital filter. 


Remark Jf k = 1 (the information frames are simply bits), then G(x) = T(1) = 
T(1+02 + 02? + 02? +--+) 

Ifk >1, G(x) does not allow an interpretation as a code bitstream; the notion 
of impulse response is then not really adapted. 
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Examples (1) k=1,n=3. 


(2) 


Choose 
1 1 0 
Giz) =(1]+[0]a+ [1 ] 2’. 
1 1 1 
The information bitstream aoaia2---at--: is transformed into the code bit- 
stream 
Cio C11 12 413... Cit... 
C20 Col C22 C23 Sige CDP airs 
C30 c31 c32 633 +++ C3t ... 


in the following way: Write a(x) = ao + aia + agx? +--- 

c(x) =co +e12 + e9n7 +--- 

then c(a) = G(x)a(x) = a(x)G(ax) is computed, frame by frame, in the following 
way (Cauchy formula for the coefficients of a product of formal power series): 


1 1 0 
Ct = tZo + at-181 + at-282 = at () + at-1 (°) + at—2 () 
1 1 1 


This gives in componentwise notation: 


Cit = At + At-1 
Cat = At + At-2 
c3t = at + At—1 + At-2 


Consider, for example, the information bitstream 


a0010203°°° = 10101010--- 
We obtain the following code bitstream: 
111111.--- 
Cocic2c3::-= 100000-:: 
110101.--- 
k=2,n=3. 
Choose 


10 11 01 
G(z)=[([01]+]10)2+]{ 11 J 2’. 
11 10 01 


The beginning of our information bitstream will be 


110011--- 


Pe =O A000 es 


1.€. 


a)=(4)+(t)2+(L)a2#(S)or+ (S)et+ (Gere 


The frames of the code bitstream e(x) = G(x)a(x) = co +eia+eox? +e32°+4--- 
are computed by means of the convolution relation 


10 11 01 
ce = Goat + Giar-1 + Goar-2 = | O1 Jae + {10 ] a1 +] 11 | ag_e. 
11 10 01 
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So, we obtain 


and so on... 
Our code bitstream will then be: 


10000--- 
Coc1e2¢3C4::- = OOLOIL-:-: 
11010--- 


Attention When applying the formulas for the first values of t, we suppose tacitly 
that every az, for t < 0, is zero. 


The (n,1) convolutional codes: Some polynomial algebra 


Recall: A (n, 1) convolutional code is obtained by means of a generator polynomial 


G(x) = go + g1z4 gor +--+ gma, 


which transforms (by polynomial multiplication) information bitstreams, presented 
as binary formal power series (a frame = one bit) into code bitstreams, presented as 
formal power series with vector coefficients (a frame = n bits). 

Write G(x) more explicitely: 


G10 gu al gi(z) 
920 g21 g2(x) 
G(z) = +[ . fate--+] 2m Po™= 
n n , nL 
Jno Jn1 Geis In(x) 
with 
g(x) = giot gira gi2x” sot game”, 
g2(x) = geo + gaix + g22u" +++++ gom2™, 
gn(x) = gno + gnit + gn2v” +--+ + gnme™. 
Thus we have: 
a(x) 1 (x) 
a(x) g2(x) 
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A (n, 1) convolutional code is defined by n usual generator polynomials which 
act simultaneously (by binary polynomial multiplication) on the same infor- 
mation bitstream (written as a binary formal power series). 


Example 
1 1 0 gi(x) 1+a 
G(x) ={1)+]0)a+]1 v= g2(z) | = 1l+2? 
1 1 1 g3(x) ltaet+a2? 
Then let a(x) = ao + a12-4 aon? +--» tax’ +--+ be the information bitstream, 


in formal power series notation. The first bit of every code frame is obtained by the 
convolution a(x)(1+ 2). 

The second bit of every code frame is obtained by the convolution a(x)(1+ 2”). 
The third bit of every code frame is obtained by the convolution a(x)(1 ++ 2”). 

The ensuing natural question is the following: Are there algebraic relations be- 
tween the polynomials gi (x), g2(x),...,9n(a) which have an influence on the quality 
of the convolutional coding? The answer is yes. 

Before giving details, let us recall some simple facts: 

A convolutional code (i.e. the collection of its words) is linear : The sum of two 
code words is a code word (addition of binary sequences: position by position — 
across the frames). This stems from the fact that the multiplication by G(z) is a 
linear operation on the information sequences. A transmission error is formalized as 
in the case of polynomial codes: 


Then let 
V = vovi1v2Vv3::: = the stream of received frames 
c = coc1c2c3--- = the transmitted code bitstream (as produced by the encoder) 
e = ege e2e3--- = the error bitstream (written as a sequence of frames) 
elt 
cot the code frame cz has undergone an error of the type e¢ 
e = . 0 this position has not been affected 
. Cet = F ages ore 
1 this position is in error 
ent 
Notation in formal power series: v(x) = e(x) + e(x) 


Look now at interesting algebraic relations between the n binary polynomials 


g(x), g2(x), tek ,Gn(2). 
ate) = (55) = (235): 


pgcd(gi(x), g2(x)) =1+2. 


Example Consider 
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Let wu = 11111111--- be the constant sequence which gives the formal power 
series: u(x) = Do a2? =1+a+2?>+2°+--- u(x) = i asa formal power series 


~~ [+a 
over Fz = {0,1}.? 


Hence: u(x)G(a) = Gare a ee 


u(x) g2(x) x 
In other words: Our convolutional encoder transforms u = 11111111... into 


11000... 
01000... 


The consequence: If after a transmission only the first bit of co and the two bits 
of c, are in error (i.e. only three bits are in error), then this corresponds altogether 
to a total bit-inversion affecting all positions of the information bitstream. Such a 
convolutional encoder is called catastrophic. 

A non-catastrophic convolutional code will thus be characterized by the following 
coprimality condition: pgcd(gi(x), g2(x),.-.,gn(x)) = 1. 


l+a 
G(x) = 142? 
l+aot+2? 


defines a non-catastrophic convolutional code (since g3(x) = 1+. 2+ 2? is irreducible 
over F2). 

How can we make a clever use of the foregoing coprimality condition? 

The answer comes from an algebraic result which is already familiar to us: recall 
the question of multiplicative inversion in the rings of residues. 


Example 


Lemma pgcd(gi(x), 92(@),---;gn(%)) = 1 => There exist polynomials 
bi (a), b2(a),...,bn(x) € Fela] with bi (x)gi (a) + bo(x)go(a) +--+ + bn(x)gn(x) = 1. 


In the case n = 2 the two polynomials 6; (x) and b2(x) are obtained by the Euclidean 
algorithm which yields the gcd(gi(x), g2(x)); the case n > 2 is easily reducible — by 
an evident recursion scheme — to the case n = 2. 


Example 
g(t)=1+2, go(x)=14+27, gs(r)=1+2+42”. 
Here, we have gi(x) + go(x) + g3(x) = 1. Thus, bi(a) = be(x) = b3(x) = 1. 
All this allows an easy reconstruction of the information bitstream in function 
of the code bitstream: Write 


c1(x) 
c2(x) 
ee)=] |, 


Cn(x) 


where 
the formal power series which describes the 


cj(x) = bitstream of the jth position of the code 
frames, across the frames 1 < j <n 


? i.e. u(x) and v(x) = 1+ are multiplicative inverses. 
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cn (#) = a(@)gn(2x). 
Then a(x) = 61(@)ci(a) + b2(x)c2(x) +--+ + bn(x)en(2). 


In our foregoing example we get immediately: The information bitstream is obtained 
from the code bitstream as follows: compute, for every code frame, the sum of the 
frame-bits (in XOR, of course); this gives the corresponding information bit! 


Attention Our arguments are “error free” — we assume a correct transmission. 


Exercises 


(1) Let C be the convolutional code defined by 


l+z2 
G(x) = 1+2? : 
l+a2+2? 


You receive the following stream of frames: 


10010... 
V=Vvovivev3sv4a::-=]{ 01011... ]. 


10101... 


There must be a transmission error. Why? 
(2) Which of the oe convolutional codes are catastrophic? 


(a) g(x) = g2(x) =1l+a x, 
(b) g kar +27, g2(x) =1l+a et+et, 
(c) g iz) =1+ a+ 2° +24, g(x) =1ta? +24, 
(d) a (z) =1+a*+a°+2°%, g(x) =14+e4a? 42°, 


(3) Let C be the convolutional code defined by 


fan) \ ff 14s 
oo) = C3) 7 fae . 
(a) Find 61 (2), b2(a) with b1(x)gi (x) + b2(x)g2(x) = 1. 
(b) Recover the six first information bits aoa1az2a3a4a5 from 


= Peet 
2S Nak ise ays 


(4) Let C be the convolutional code defined by 


gi(x) l4+e?423 
ace)= (sat ) = ( l+et+a2° 
g3(x) ltata?4+a3 


(a) Find b1(2), b2(x), b3(a) with bi (x)gi (x) + b2(x)g2(x) + b3(x)g3(x) = 1. 
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(b) Recover the six first information bits apa,a2a3a4a5 from 


100000... 
C= cocicec3c4c5--- = | 111100... 
110111... 


Towards Error-Detection and Error-Correction: The Minimum 
Distance d* Between the Words of a Convolutional Code 


Formally, an (n, 1) convolutional code C is a set of sequences ¢ = CociC2c¢3... over 
the alphabet of the 2” binary frames of length n. 

The generation of C by means of an encoder of the form c(x) = a(x)G(a 
(language of formal power series) has two consequences: 


(1) C is linear, ie. it is stable by addition (of infinite binary matrices with n 
lines: we act via Boolean XOR, binary position by binary position). 


(2) C is stable under “annihilation of leading zero frames” : 
if c = cocice2c3:-- EC and if cp is the zero vector, then cc’ = cic2c3:-: EC. 
(note that — conceptually — this is merely a time-unit shift ...) 


The argument which gives (2) is the following: 


G(x) = go + gixt+ goer? +---+ gma” 


and go is not the zero vector. 
Hence, if c(x) = a(x)G(a), then 


0 
0 
co=]. = =a =0. 

0 
We get: 
Ci = a180, 
C2 = a280 + a1g1, 
C3 = A380 + 4281 + 4182. 
Thus, ¢/(#) =c1 + cor +e3@7 +--+ = (a1 tagrt...)G(ax) €C. 


It is because of property (2) that we will often assume, in primarily theoretical 
considerations, that the code words in question should begin with a non-zero frame. 

As in the case of ordinary polynomial codes, the encoder — given by the mul- 
tiplicative action of the generator polynomial G(x) on the various information bit- 
streams — must create distances between the code words: In order to make sure that 
a (decent) transmission error can be detected (and maybe even corrected), it must 
above all be impossible that such a transmission error transforms a correct word 
(the transmitted code word) into a different correct word (the received word). The 
correct words (the code words) have to be “sufficiently distant” of one another. This 
explains the interest of the minimum distance between the code words. 
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In the situation of the convolutional codes, we are confronted with words of (the- 
oretically) infinite length. This does not change our usual arguments: The distance 
between two words (i.e. the number of binary positions where they are distinct) is 
precisely the weight (= the number of non-zero binary positions) of the word which 
is their sum. A convolutional code being linear, the minimum distance do. between 
its words is nothing but the minimum weight of its non-zero words (beginning with 
a non-zero frame). 

Let now G(x) = go+giv+ger?+---+gma™ be the generator polynomial of the 
considered convolutional code C. G(x) € C, i.e. the word G = gogig2--:gm000--- 
is a code word in C. 

The weight of G : The number of non-zero positions of the (vector) coefficients 
of G(x). Hence: da. < the weight of G. For the decoding arguments (error-detection 
and error-correction) we need another notion of minimum distance. It is clear that 
a practicable decoding algorithm only can work by means of an inspection window 
(of fixed finite length) moving over the bitstream of received frames, which, itself, 
will be — at least in principle — unlimited. 

We guess that we should be able to correct “dispersed” transmission errors of the 
following kind: The number of binary errors per inspectable sector does not exceed 
a certain threshold. We shall see in a moment that here the “natural” minimum 
distance between the code words will be the following: 


d* = minfdmii(c,c’) : ce’ € C,eo 4 c’o} |m=degG(x)} where 
dm+i(e, ce’) = the number of binary positions where the two initial segments 
CoC1C2...Cm and cpe{ces...c), of c and c’ are distinct 


d* is then the minimum weight of the m+ 1 first frames cocic2...Cm for all 
code words c beginning with a non-zero frame. 

Since G = gogige... 8m (the coefficients of G(«)) is in the competition, we will 
have d* < the weight of G = gogigo... 2m. 


Example 
1l+2 1 i: 0 
aia)= ( 1+? )-(Q) (Je¥Q)* 
l+a2+2” 1 1 1 
110 
The weight of G = gogig2 = (: 0 ' is equal to 7. 
111 


We search for d* and dy. 
We know already: d* < dx < 7. In order to find d*, we need to consider all 
“prefixes” of three frames (m = 2) for code words which begin with a non-zero frame 


(i.e. with 
1 
go = 1, ) 
1, 


which is the only possible non-zero beginning for a code word). 
So, let us compute c = (1+ aix7 + azx”)G(a) for aiaz2 = 00,01,10 and 11. 


110000... 
(00) c={101000... ) the weight ofc:7; the 3-weight ofc: 7. 
111000... 
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111100... 
(01 c={100010... | the weight of c: 10; the 3-weight of c: 6. 
110110... 


101000... 
(10 c={111100... } the weight of c: 8; the 3-weight of c: 6. 
100100... 


100100... 
(11 c= € 10110 . the weight of c: 9; the 3-weight of c: 5. 
101010... 
We see: d* = 5 and dw~ = 7. 
(Note that we have necessarily: d.. > d*+ the weight of the highest coefficient 
of G(x).) 


Exercises 


(1) Find d* and dx for the convolutional code given by the generator polynomial 


Gay = ( ltet+a ) 


l+ote? +2? 


(2) Find d* and da for the convolutional code given by the generator polynomial 


1+? 
G(x) = | l4+a+a? |. 
lt+a+2? 


(Here, g2(x) = g3(x); this is permitted and usual...). 
(3) Find d* and d. for the convolutional code given by the generator polynomial 


Ll+a?+2° 
lta+a? 
G(x) > il r fod 


l+ota?te? 


(4) Once more the code of the preceding exercise: 


1 
ce ; e+ 
1 


eo 
w 


1 
@(e)=| 5 |+ 
1 


ae) 


1 


Preliminary observation: The first frame co of a code word c(x) is either the 


word zero or go, the constant term of G(a), according to the alternative { 1 
for the initial bit ao of the information word a(z). 
Moreover, (1+ a1a+agx?+...)G(x) = G(x) + (ara + agx? +...)G(x). Taking 
this observation into account, find aoga1azga3a4a5 — the first six bits of the 
information bitstream — which give the following code bitstream: 
111100110... 
101110011... 
~ 101110011... 
100000000... 


C = CoC 1 C2C3C4C5CEC7Cg8°:: 
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(5) (Refrain from too optimistic decoding). Consider the convolutional code, given 


by 
1l+2 1 1: 0 
G(x) = 1+2? =({1)4+ [0])a+ [1]. 
l+a4+2? 1 1 1 


[We know: The first frame co of an error free code bitstream c is either 


0 1 
(°) (for a9 = 0) or () (for ao = 1).] 
0 it 


We receive: 
0001000... 
v=0101100... 
0010100... 


Are we allowed to conclude that the information bitstream begins with ao = 0? 


(a) Compute e (x) = (1+ «+ a”)G(a) (corresponding to the information bit- 
stream agaia2--- = 11100---) and determine d(v,c) = the number of binary 


positions where v and c™ are distinct. 
(b) Show that d(v,c) > d(v,c“) for every code word ¢ 4 c. 


The Down-to-Earth Decoding 
Some Simple Formulas 


Notation In the sequel, the derivation sign will be used to indicate a simple left 


shift (with annihilation of the constant term): (ao + aix + agx” + a32°4+...)! = 


ai +aor+ag3r?+.... 
Settle in the following situation: 


gi(z) 
G(x) = : =gotgirc gox" +... Bmx 
Jn(2) 


the generator polynomial of an (n,1) convolutional code. 
The information bitstream aoaia2...az... is transformed into the code bit- 


stream Cocic2...c¢... by “binary convolution”: | e(a) = a(x)G(z) 


a(x) =angtaxtaor?+---+aa' +... 
where 


2 t 
ce(x) = co Fea + cgn* +--+ Eu" +... 


The code bitstream will pass through a channel (in the broad sense), which will 
give rise to a bitstream of frames eventually in error: 


VoVivV2...Vt..- 
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The transmission error is described by an error bitstream 
€pe€1€2...€¢..., 


which simply adds (Boolean XOR, binary position by binary position) to the code 
bitstream. 


With v(@) = vo tviat+ ver +---4 vinci +... 
e(x) = €o eyn tenor? +---tea t+... 
thus we get: v(x) = c(a) + e(az) 


Suppose that you have decoded the first bit a9 by an algorithm which recovers 
the beginning of the information bitstream from (the beginning of) the stream of 
the received frames — under the condition that the number of transmission errors is 
decent, of course. 


How to reinitialize the situation? 


c1(x) = a'(x)G(x) = (a1 + aan + 3x? +...)(go + gie + gor? +---+ emer) 


= = (colt) + a0 Gla) = <(v(@) + ele) + a0 - G(x) 


Now: Vo = dogo + €0; hence ¢;(x) = v'(x) + e’(x) + ao: G(x) 


Put | vi(x) =v’ (x) + a0- G'(z) 


then 


We insist: 


ao = 0: €1(x) = e6(a) = v(x) + e'(2), 


ao = 1: e1(2) = {v'(a) + G'(x)} + e'(2). 


In the case where ag = 1, the bitstream of the received frames will thus be reinitial- 
ized to |v1 + gilv2 + gol... |Vm + Sm|Vm4i]Vm+z2|--- 

Thus, in a step-by-step decoding, with iterated reinitialization, the sequence 
of the received frames will be always readjusted according to the “arithmetical ra- 
diation” of the first bit (= 1) over the block of the m+ 1 leading frames. Of course, 
all this can easily be generalized. 
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Exercises 


(1) 


Suppose that we have decoded — in a single step — the first four bits ao, a1, a2, 
a3 of the information bitstream. Consider the corresponding reinitialization. 
Let ca(x) = a (x)G(x) = (aatasx+agx?+...)(go+gia+gen?+---+gm2™). 
Show that ca(x) = va(x) + e (a), where va(x) = {v(x) + (ao + aia + agar? + 
a3x°)G(a«)}, 

Still the situation of the preceding exercise. 

Let thus aoaiaz2a3 be the (decoded) prefix of an information bitstream such that 
the corresponding bitstream of received frames is V = voViv2...Vi..- 

Let w = wowiwe...w;... be the reinitialized stream of received frames: 
va(x) = Wot wit 4 wor? +wa0' +... 

These will be our fixed data. Consider now all information bitstreams with prefix 
aoaiaza3. For such a sequence aoa1az2a3a4a5a¢... and for N > 5 let e¢y_1) = 
Coc1...cn_—1 be the segment of the N first frames of the code bitstream, and 
€(n—5) = Co€1...€n—5 be the segment of the N — 4 first frames of the code 
bitstream associated with asasag...: ca(x) = (aa tase t+ agar? +...)(go+ 
git t+ gor? +-+-+gme™) =O +r+eor’ +... 


Show that then 


d(€wv—s), W(w—5)) = d(ecw—1), Viw-1)) — —A(€(3), Vay) 


(We note that the term d(c(3),v(3)) only depends on the common prefix 
aoaia2a3, and is thus a constant of our problem ...) 


The Decoding 


Recall, once more, the formulary for our decoding situation: 


So, let 
gi(z) 
G(x) = : =go+gixt gor +--+ 8mx™ 
gn (x) 


be the generator polynomial of an (n,1) convolutional code. 


The information bitstream aga,a2...a;--- will be transformed into the code 


bitstream cocic2---cz--- according to the familiar operation 


e(x) = a(z)G(z), 


a(x) = ao + ara + aon? +---+ art +--- 


where 


c(z) = co tere + eon? +---+ era? +... 
The bitstream of received frames (eventually in error): 


Voviv2...Vt-°°: 


The error bitstream epe1e2---e:--- adds simply to the code bitstream. Put 
v(z) =votvie4 vor? +--+ +viat +t... 
e(x) =eo tera tegr?+---+ea'+--- 
which gives v(x) = c(x) + e(z) 
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The decoder has to reconstruct the correct information bitstream aja ,a2...a;... 
from the stream of received frames Voviv2...vzi... under the condition that the 
error bitstream eoeie2...e¢... is “locally light”. 

More precisely: Let m = the degree of G(x), and let d* be the minimum distance 
of the convolutional code. Make a window of m+ 1 successive frames move along 
the error bitstream. If the number of binary errors over m+ 1 successive frames 
never exceeds the threshold 4(d* — 1), then the decoding algorithm will correctly 
reconstruct the initial information bitstream. The decoding is done (in the worst 
case) bit by bit. 

Compute, for every binary word bobi...bm of length m+ 1, the m + 1 first 
frames of its code word: c(bob1...bm)(m) = Co€i...€m then compute, in each 
case, the distance towards the m+ 1 first frames of the received word : d = 
d(€o€1 ...€m,VOV1..- Vm): 

According to our hypothesis (on the number of binary errors per inspection 
window of length m +1), the correct prefix aoa...@m of the current information 
bitstream will give d = d(coc1...¢m,VoV1--.Vm) < +(d* — 1). Let now aga} ...ai, 
be another segment of length m+ 1 such that d’ = d(cgc,...Cim,VoVi-.-Vm) < 
3(d* — 1). Then d(coc}...cj,,€0C1-..Cm) < d* — 1. This means cy = Cp, ice. 
a9 = ao (go will always be the constant vector of n values 1). 

Let us sum up: If the transmission errors are weak in the sense that the number of 
binary errors over m-+1 successive received frames never exceeds the value +(d* —1), 
then every binary word agai ...@j, with d’ = d(coc} ... Cin, VoV1... Vm) < $(d*—1) 
begins with the first bit of the correct information bitstream: aj = ao. One saves the 
first bit which has been identified this way, and one reinitializes for the next step. 


Remark Whenever there is only one single word aoai...@m such that d = 
d(coc1...Cm,VoV1--.-Vm) < 4(d* — 1), then it is the right beginning of the 
information bitstream, and we can save the entire block of the m+ 1 bits, then 
reinitialize according to the method discussed above. 

We shall see examples in the following exercises. 


Exercises 


(1) Consider G(x) = (ie) = + (Q)=+ GG 


(a) Show that d* = 3. We can thus decode correctly provided there is maximally 
one single binary error over three successive received frames. 
(b) Consider the first 16 frames of the received bitstream : 


Vo V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 


0o1i1%1i131d31031id11 21 «231 «02121 
1003100031101 1 1 «0 0 1 
Find the first 12 information bits apa ,a2...ai1.- 
l+a2?4+23 1 0 1 1 
a l+2+2° Pa 1 0} » i (ae 
(2) Let G(x) = {eee =l4 a 1 ct 9 | + i|* 
ltata?4+a3 1 1 1 1 
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(a) Show that d* = 7. 
We will decode correctly if there are not more than three binary errors over 
four successive received frames. 

(b) Look at the first 20 frames of the received bitstream : 


Vo V1 V2 V3 V4 V5 V6 V7 V8 V9 Vio V11 V12 V13 V14 V15 V16 V17 Vis Vi9 


100010031100 1 1 «21 0 1 0 0 1 =0 
0111131000211 «1 «21 0 0 1 0 1 41 ~0 
0111110000021 1 «21 0 1 0 1 21 =«0 
0101111011071 «21 «21 «0 0 1 0 1 +0 


Find the first 12 information bits agaia2...ai1. 

(3) In the two preceding exercises, the decoding can be done in blocks of m+ 1 bits 
(i.e. in blocks of 3 and 4 bits, respectively). This is always possible for these two 
codes. The reason? 

(4) Consider, once more, the convolutional code of exercise (2). Let now the stream 
of the received frames be the following: 


Vo V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 


eFoco 
oocrF 
rococo 
Orr Fe 
ooo O° 
OrRrREH 
OrRrREH 
OrRrREH 
(a = om ae) 
OrRrREH 
OrRrRE 
Orr re 
ooo oO 
OrRrREH 
OrRrREH 
Orr eH 
ooo o 


Find the first 12 information bits agaia2...ai1. 


4.2.2 Decoding: The Viterbi Method 


The Viterbi decoder is an ambitious (and universally acclaimed) attempt to replace 
the dull simplicity of the down-to-earth decoding by a more flamboyant algorithmic 
approach. 

What is the gain? In sufficiently decent situations, we will decode — in a single 
step of the algorithm — rather long prefixes of the information bitstream (but we 
will have to invest time and space). 

What is the loss? The mathematical control of the situation needs slightly 
stronger hypotheses than the down-to-earth decoding. 

Recall, once more, the decoding situation: 


Voviv2...Vi... = the bitstream of the received frames — eventually in error, 
epe1e2...e:... = the error bitstream, 
CociC2...cz... = the code bitstream. 


We have: v(x) = c(a) +e(x) (formal power series notation). The code bitstream 
is generated from the information bitstream agaia2...a,... by “arithmetical filter- 
ing”: c(a) = a(x)G(ax) (formal power series notation), 
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where 
g(x) 
G(«) = =gotgit+ gor +-+-gmx 
n(x) 
is the generator polynomial of the (n, 1) convolutional code. 
The Viterbi decoder should reconstruct the correct information bitstream 
aoaia2...azr... from the bitstream of received frames voviv2:::Vvi-:: under the 
condition that the error bitstream epe1e2...e¢... is “locally thin”. 
Competing with the down-to-earth decoder, we want the familiar condition: 
Let m = the degree of G(a), and let d* be the minimum distance of the consid- 
ered convolutional code. If the number of binary errors over m+ 1 successive frames 
does not exceed the threshold $(d* — 1), then the Viterbi algorithm will correctly 
reconstruct the initial information bitstream. 
But we shall encounter some (minor) problems. In order to understand the com- 
plications, let us look first at the structure of the algorithm. 


— The interior algorithm establishes — in widening step-by-step the horizon of 
evaluation — a list of binary words which are the candidates for the beginning 
of the information bitstream. 

— The exterior algorithm chooses the common prefiz of the candidates as the be- 
ginning of the decoded information bitstream; then, it reinitializes according to 
the well-known procedure. 


The correct functioning of the algorithm is guaranteed whenever: 


(1) The beginning of the correct information bitstream is actually in the list of the 
candidates of the interior algorithm; 
(2) There is always a common prefix of the candidates. 


As we have already underlined, we would like that the hypotheses which allow 
the decoding by the down-to-earth algorithm guarantee similarly the decoding by 
the Viterbi algorithm. Alas, this is not exactly true.? 


The Interior Viterbi Algorithm 


Fix first the length N of an “inspection window”. N will be an appropriate multiple 
of n, i.e. of the number of components of the code bitstream. Often N = 2n will be 
sufficient.* 

Recall: m = deg G(z). 

The interior algorithm associates with voviv2...vnw-—1 (the N first received 
frames) a list of (at least) 2” initial segments aoa1a2...an—1 of the possible in- 
formation bitstream, which are the candidates (the most probable segments) for a 
decoding of voviv2...vn—1i. What is the logic of this list? 


3 For the practitioner, these problems don’t exist: A deficiency of the decoding 
algorithm for him always means that there were too many transmission errors — 
and he is not wrong. 

4 This strange appearance of n seems to come from a visualization which concate- 
nates horizontal frames. 
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One would like to find the correct beginning of the information bitstream. 
Hence: The m last binary positions an-m...an—1 of the candidates will be free 
and thus without significance for the decoding. The N — m first binary positions 
ao@1...@N—m—1 are more and more significant towards the head, and are deter- 
mined by an “iterated bifurcation” algorithm. Step by step — always in function of 
the m last (parametrizing) positions — one chooses one of the two alternatives for a 
test segment, in comparing the distances of the associated code segments towards 
the current segment of the received bitstream . 

So, we shall obtain a final list of (at least) 2” candidates, which comes up this 
way: For every intermediate length L (m < L < N) we establish a list of 2” + ez 
preliminary candidates aoaia2...az—1 them last positions of which will be free 
(non-significant) and the L — m first positions of which will be significant for the 
probable beginning of the original information bitstream. The term ez, stems from 
the fact that the algorithm is sometimes forced to accept the two possibilities of an 
alternative where it should make a choice — an event which hopefully will become 
rare towards the end of a round. 


Description of the Interior Algorithm 


Step no 1: 
Every binary word aia2...d@m is a terminal segment of two information words: 
Oa1a2...@m, laia2...dm. Let us determine, in each case, the most probable 
segment (with respect to our decoding situation): 


(1) Compute 
c° = the m +1 first frames of the code word of 0+ a,x + aox? +-+>+Qm2z™ 
c! = the m+ 1 first frames of the code word of 1+ air + aox7 ++: +dmx™. 
(2) Let vim) = VoV1V2-..Vm. Compute 
d° = (v(m); c°) = the number of binary positions where Vim) and c° are distinct 
d' = d(V(m),¢€') 
(3) Save 
Oaia2...dm if d° <d' 
laja2...dm if d°>d! 
both if d =d' 


We obtain thus 2” + €, segments apa1...@m ( parametrized by the last m bits). 
The term €; comes from the case without exclusion. 

Step no 2: 

Every binary word azqa3...@mQm+41 (of length m) gives rise to two types of 
possible prolongations, in function of the choices of step no 1: 

ao0a2a3...AmAm+1 aolaza3...AmAm+1 

Note that ao is determined by 0az2a3...@m (and by laga3...am) according to 
the preceding step (and will not necessarily have the same value in these two cases — 
you can also have “multiplicities”, due to non-exclusions in the preceding step). Let 
us find, for each of the m-tuples az2a3 ---G@mG@m-+1 the most probable prolongation: 


(1) Compute 


c° = the m +2 first frames of the code word of 
do + gn? +++» +amae™ + Om4i2™*", 
c! = the m +2 first frames of the code word of 


ag ta aon? +--+ +am2™ 4 Gmaiet?. 
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(2) Let = Vim41) = VoViV2---Vm41. Compute d° = d(Vim41),c°) d’ = 
A(V(m-+1)s c’). 
ao0a2...Am4i1_ if d° < d! 
(3) Save aolaz...am4i if d>d' 
both if d°=d'. 


We obtain 2” + €2 segments aoa1a2...@m+1 (parametrized by the last m bits). 

aoa is determined by az2a3...A@m@m+41- 

Step no tf: 

Consider the 2” binary words atat+1...@m+t—2Am+t-1 (of m bits each). Ac- 
cording to the choices of the previous steps, each of these words gives rise to two 
types of information words which extend the preliminary candidates on the list of 
the preceding step: 

ao... at—20az -+-Am+t—2A4m+t—-1, Go... ar—2lazt «++ Am+t—2A4m+t-1 

Note that the prefix aoai...at—2 is a function of the segment tae . 6. Om4+t—2 
according to the choices of the preceding step (attention to possible “multiplicities” , 
due to former non-exclusions). 

Let us find, for each of the m-tuples azai+1 +++ @m+t—2Am+t—1 the most probable 
prolongation (in the sense of a logical decoding): 


wee the m +t first frames of the code word of 
~ ao tes: + ar-ox?? + ape’ +--+ + amqr-ia™ 
(1) Compute 
; _ the m +t first frames of the code word of 
~ ao tess Faron? + 2°! + aya’ +++) + amye-ie™ | 


(2) Let = V(m4t-1) = VoViV2...Vm4t—-1. Compute 


d? = d(v(m+t—1),€°) 
d= A(V(m+t—-1); c') 


(3) Save 


ao... @r-20at.--@m4t—1 if d° < d! 
ao... @t-21at..-@m4t—1 if d° > d! 
both if d°=d' 


We obtain 2" +e; segments aoa1d2...@m+t—1 (parametrized by the m last bits). 
aoa1...at—1 is determined by azt...@m+t-1- 


End of a Round of the Interior Algorithm 


After N — m steps, we hope to obtain (only) 2” information words aoai...a@n-1: 


rene! 1 
ara ...aW,, 


a?) a) oOn 


ae 


j gm gre 2 
aa?” a2”) 


the prefixes of which are the candidates for the beginning of the original information 
bitstream. We note that a generous choice of N and a decent error distribution will 
probably make the cases of non-exclusion disappear in the last step. 
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l+a fl, 1 0 
09-(122.)-()-Q)()e 
l+at+ae’? 1 1 1 


The formula for the computation of the tth code frame is then: 


1 1 0 
ce =aze| 1 | +ae-1{ 0] +a:-2{ 1]. 
1 1 1 


Now the first eight frames of the received bitstream: 


Example 


Vo Vi V2 V3 V4 V5 V6 V7 


ao 


hod of. de ke 1 
100 0 0 0 
1 11010 


FOr 


We shall fix N = 3n=9. 


Convention We note that the Hamming distance” is additive in the following 
sense: 


A(V(e4+1), C(k+1)) = A(V(K) Va +1, C(k)CK+1) = A(V(K), C(K)) + A(Ve41, Ch41)- 


This will permit us to write, for example: 


c= (4)1 
0 
We have replaced all the frames of c, except the last, by their distance from the 
corresponding segment of v. The distance of c from the entire segment of v will then 
0 
be equal to 4 + the number of positions where 1 differs from the terminal frame of 


0 
the current segment of v. 


Step no 1: 

000 
silver c°=000 d?=8 
0 a1 a2 000 
000 100 survives 
100 tao 

c'=101 d'=3 
si baat 


5 . . . 
° Indeed, our innocent distance between binary words (vectors) has a name! 
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001 
maa © 700! &=7 
er ey 001 
001 101 survives 
120 aes 
c'=100 d'=2 
110 
011 
ao a1 a ce =010 d=3 
aan ta 011 
010 010 survives 
110 101 
c'=111 d'=4 
100 
010 
ree c&=011 d=6 
ees 010 
o1t 111 survives 
111 100 
c=110 d=3 
101 
Step no 2: 
0 
e='6)0 @=5 
ao a1 42 a3 0 
1000 : |We save both| 
04100 c' = (3) 1 d=5 
1 
1 
c? = (3) 1 gd =4 
ao a, a2 a3 1 
1001 , |We save both 
0101 c' = (3) 0 d—4 
0 
1 
c= (2)0 d=2 
ao a1 a2 a3 1 
10410 7 1010 survives 
11410 


c= (3)1 db=5 
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ao a1 a2 a3 


1111 survives 


1oi11 0 
ine c'=(3)0 d'=4 
1 

Step no 3: 

0 
ao a1 ag az3a4 c°= (5)0 d?°=6 

0 
1000 0 |10100 survives 
01000 0 
10100 e¢=(2)1 d=5 

1 

1 
ao a1 ag a3aa c?=(5)1 d°=7 

1 
10001 |10101 survives 
01001 1 
10101 e=(2)0 d=2 

0 

1 
ao @, ag az3a4 c?=(4)0 d=5 

1 
1001 0 | We save the three 
01010 1 
11110 e=(41 d=5 

0 

0 
ao Q1 a2 a3 a4 co = (4) 1 d° =6 

0 
10011 |We save the three 
01011 0 
11111 +ec=(4)0 d'=6 

1 
Step no 4: 
do @1 G2 a3 aaa 
0 G1 G2 a3 G4 a5 c= (5)0 d®=7 

0 

101000 
100100 : |We save the four 
010100 fs 1 
re ae ee Bo se ee 
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d° =6 


ao G41 G2 a3 a4 a5 


|We save the four| 


101001 


100101 


0101041 


11141041 


c= (2)0 d°=2 


ao G1 G2 a3 a4 a5 


|101010 survives| 


1031010 


100110 


010110 


113141410 


ao G1 G2 a3 a4 a5 


|101011 survives| 


10310141 


100111 


0103111 


111414141 


Step no 5: 


ao G1 A2 43 a4 A5 6 


101000 0 


1 | 1010100 survives | 


1001000 
0101000 


d' =5 


c' = (2) 1 


11131000 


10310100 


ao 41 G2 43 a4 A5 6 


1010001 
1001001 
0101001 


|1010101 survives | 


(2)0 d=2 


ci 


11131001 
1010101 


1 
c° = (6)0 d=7 


ao G1 G2 43 a4 A5 6 


1 


101001 0 


|1010110 survives | 


100101 0 


0101010 


d' =6 


ec! = (5) 1 


111310410 


10310110 


ao G1 G2 43 a4 A5 6 


1010011 


|1010111 survives | 


1001011 
010310141 


c= (5)0 d=7 


11i11o0i1i4t1 


1010111 
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We observe: After five steps, there remain correctly 4 (=2”", for m = 2) (prelim- 
inary) candidates aoa1a2a3a4a5a¢ for the actual prefix of the information bitstream. 
Moreover, they have a common prefix (of maximal length), i.e. aoaiaza3a4 = 10101. 

Once more: The four terminal couples asa¢ are of no significance for the decod- 
ing; they only enumerate our candidates. 


Exercises 


(1) Carry out the sixth step of our example above. The list of candidates that you 
will obtain is the following: 10101000 10101010 10101001 10101011 

(2) Show that a common prefix of all the (preliminary) candidates produced by a 
step of the interior algorithm will remain a common prefix for all the following 
steps. 


We shall take up again three decoding examples that have been treated at the end 
of the preceding section — but now in the context of the Viterbi algorithm. 


nae _f 1+2? \_ fil 0 dN, is 
(3) pesin with G02) =( } 272 )=(1)+()e+ (i) and 


Vo V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 


011111210 1 1 1 01 1 
10010001 0 1 1 0 0 1 


Carry out the first ten steps of the Viterbi algorithm and recover the common 
prefix of the candidates in the final list. 
[Answer: The common prefix: 1100] 


1 1 
1 1 


(4) Then 
l+a7?4+23 1 0 1 1 
_ lt+2+2° me 1 O| 2 1) 3 
G(x) eee: St hb Pog RE ooh earl ge 
l+ate?4+e3 1 1 1 1 


as well as the first 20 frames of the received bitstream: 


Vo V1 V2 V3 V4 V5 V6 V7 Vg V9 Vio Vi1 Vi2 Vi3 V14 Vi5 Vie Viz7 Vig Vig 


al 
0 
0 


rer Oo 
ero 


1 
1 
ab 


PRR oO 
RPrRrRO 
n=) 
oocorF 
Foor 
Foro 
on ee a) 
PR RR 
Pe RR 

PR 
Le 
Orro 
PR RR 

ooo 


01011 0 O 0 


Go through the Viterbi algorithm, step-by-step, until the list of candidates re- 
veals a common prefix. 
[Answer: You need 16 steps. The common prefix: 101100111000] 

(5) Consider, once more, the convolutional code of exercise (4). A constant infor- 
mation bitstream agaiaz2a3--- = 1111... gives rise to an essentially constant 
code bitstream : 


5 
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Inflict a periodic error of three bits onto every fourth frame: 


€0 = €4 = €g = €12 else. 


Orr Fe 
eo) 
> 
II 
ooo oO 


This gives the following bitstream of received frames: 


Vo V1 V2 V3 V4 V5 V6 V7 V8 V9 V10 V11 V12 V13 V14 V15 V16 


rFPoOoo°e 
ooocor 
rFPoo°e 
ORFF 
ooco 
OrRrRH 
Orr rH 
Orr FH 
ooo oO 
OrRrREH 
OrRrREH 
OrRrEH 
ooo oO 
OrRrREH 
OrRrRH 
OrRrRFH 
(com I cae A am ae) 


We note: d* = 7, hence the number of binary errors over four successive frames 
is correctly bounded by $(d* — 1). 


(a) Carry out the Viterbi algorithm until the 22th step. 

(b) Show that the interior algorithm will never terminate with a list of candidates 
which have a common prefix (there will always be a candidate beginning with 
the bit 0). 


Help. You should show: Let at step n° 4k — 2, k > 3, the list of candidates 
(with their distances from v(4,)) be the following: 
a000 d—2 di100 d 
x001 d  el01l d 
b010 d_ f110 d—2 
cOll d-1 glll d 
then, at step n° 4k + 2, the list of candidates (with their distances from v(4p+)) 


will be: 
c0110000 d+1 a0001100 d+3 


Serie d+3 a0001101 d+3 

a0001010 d+3 gi1111110 d+1 
f1100011 d+2 g1111111 d4+3 

Here, a, b, c, d, e, f, g are binary words, x will be a list of binary words. If then x 

is “infected” by a prefix beginning with the bit 0, then the algorithm will never get 


rid of it. 


The Interior Algorithm and the Decoding 


Now, we want to seriously attack the following natural question: 

Let aja} ---a;--- be the original information bitstream. 

Does the initial segment ajaj---ay_, of the correct information bitstream 
appear as one of the candidates aga; ...an-—1 in the final list of the interior algo- 
rithm? 

The answer should be the following — taking in account the well-functioning of 
the down-to-earth decoding under this hypothesis: 
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Proposition Let m = deg G(z). 

If the number of binary errors over m+ 1 successive frames never exceeds the 
threshold 4(d* — 1) (where d* = the minimum distance of the convolutional code), 
then the beginning of the original information bitstream will appear in the final list 
of the interior algorithm. 


Proof Formal preliminaries: Put, for an arbitrary binary word a: 


||al| = the weight of a = the number of non-zero binary positions of a. 
We have trivially: ||a1a2|| = |/ai|| + ||a2|| (the weight is additive with respect to 
concatenation). 


The associated Hamming distance: 

d(a, b) = ||a+b|| = the number of binary positions where a and b differ (a and 
b are supposed to have the same format: two vectors in the same space over F2). 

Our situation: An (n, 1)-convolutional code C, given by its generator polynomial 
G(x) = got gig+ gor? +---+gm2™ 

We note: G = gogi...8m00--- EC 

Recall: d* = mincec{|leoc1...cm|| : co #4 O} Clearly: d* < |/G|| = 
[SoS --- Sm. 

Our data: 

The original information bitstream: ajaja3...a;... 

The associated code bitstream: cpcjcs...cz--- 

The bitstream of received frames: vov1V2...Vi--- 

The error bitstream: epe1e2...e;... 

We shall make use of the formal power series notation: a*(x),c*(x), v(x), e(z). 
c* (x) = a*(x)G(a) 
v(x) = e*(x) + e(x) 
Our hypothesis: Let e+ = |lerer41---et+m|| for t > 0. 
Then] 2e.+1<d* fort>0 


Thus we will have: 


1+2 
[Example: For G(x) = 14x? we got d* =5. 
l+a+2? 


In this case, our hypothesis becomes: There are not more than two binary errors 
over three successive frames — which have nine binary positions.] 

Let us show: Under our hypothesis (the error bitstream is “locally thin” ) 

agajaz-+-ay_y, will be in the final list of the interior algorithm. 


We will show: | agai...@m,4+-1 survives at the tth step of the algorithm. 


Recursion over t: |t = 1: 

Let agaj---a*, be the correct beginning of the information bitstream, and let 
Ga} ...ax, be its alternative: G@ = aj+1 mod 2. c* = cpcj ... cz, the initial segment 
of the correct code bitstream €* = $C] ...C*, the initial segment of the alternative 
code bitstream. 

We have: ¢* = c*+G hence: ¢* differs from c* in ||G]| positions (and ||G|| > d*). 
Now: d(c*,V(m)) = €0 (Since vov1-:-Vm = C9C}- ++ Cy, + e0€1 ++: em). 

Then: d(€*,v(m)) > €0 (and aja} ...a;, will be eliminated at the first step). 

Otherwise we would have: d* < ||G|| = d(c*, €*) < 2e0, a contradiction. 
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Finally: ajaj...a*%, | survives at the first step. 
trettl: 
Suppose that agaj ...@%,+44—1 has survived at the tth step of the algorithm. Let 
us show that then agaj...@7,44-1@m4, will survive at the (¢ + 1)th step. 

At any rate, since agjaj---a;,1,-1 is an output of the tth step, agaj--- 
Qm+t+-14@m+4- will be in the competition of the (t + 1)th step. The alternative 
bobi ...bs-1@faj.1...a7,14 is obtained, in polynomial notation, by addition of the 
polynomial 609 + 61” + box? +--+ 6:-12* 4+ 2° (note that we don’t really control the 
difference word 6061 ...6¢-11 of the two alternatives). 

Consequently: 


G* (x) = c* (x) + (50 + bia + bon +...54-10°"' + 2*)G(a). 

Consider C(n42) = €0°** C¢_-1€? +++ Cmte aNd Clmyry = CO... C716 --- C4, and 
the difference word d(m+2) = dodi ... dm+4t—-1dm+e (we have to do with the m+t+1 
frames which are the coefficients of the polynomial (60 + 61” + box? +... bp-yet th t+ 
x')G(2)). 

This yields: €(,.414) = €(m+t) + A(mtt)- 

And then: d(¢/,,41)»V(m+t) = |leoe1..-@m+e|| and, for the alternative, 


d(C(m+t): V(m+t) = ||eoe1 ...@m4t+dodi... dinsell. 


We must show: 


Under the hypothesis “||erer41 ...e++m|| < $(d* — 1) for t > 0” we get: 
||eoe1 +e Cmit + dod,... din+ell > ||eoex sited em+t| 


We shall encapsulate into our recursion a proof (by recursion on t) of the state- 
ment above. 

We note that the level ¢ of our (auxiliary) recursive structure consists of all 
binary polynomials 69 + 61” + box? +...64-12° 1 + xt of degree t. 

The beginning of the recursion (the degree t = 0) is nothing but the beginning 
of our “exterior” recursion (corresponding to the initial step t = 1). 

For t— 1+ t, write dodi...dm4: = dod) ...dii44-1 + O0¢-1)G according to 
(60 + 61a + boa? +... Opa + x')G(x) = (60 + dix 4 boa? +... 64-12" !)G(a) + 
a'G(a). 

We have clearly: 

d(dodi ...dm+t, dod) -.-dinyi-1) = d(didipi..-dm4t, didiy1.--dinge_1) = 
|G 

On the other hand: 

d(eoe1..-@m+t,d0d)---dinai-1) = d(eoer...em4t—-1, dod)... dinge—1) + 
lem++|| > ||eoe1 ...e@m-+z|| by the recursion hypothesis. 

If we replace the dj, by the dz, we change, beginning with the index t, ||G|| 
binary positions of the last m+ 1 frames. But, er€:41...@m++ admits e¢ < 4(d* —1) 
non-zero binary positions. Its additive partner changes by addition of ||G|| > 2e 
1-positions. Hence: 

llerer44 -++Cm+t + didi+i vod din+ell > llerer44 +++ Omit > didiii acest Greil, 
which yields our claim. 
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(The attentive reader will note that the recursion hypothesis of the “techni- 
cal” part is independent of the recursion hypothesis of the “conceptual” part: 
dod ...djn4:-1 has nothing to do with the promotion of aja} ...ay,41~1 at the 
preceding step). 


The Exterior Viterbi Algorithm 


The exterior Viterbi algorithm receives at the end of each round of the interior 
algorithm its list of candidates (for the beginning of the information bitstream), 
chooses the common prefiz of the candidates and declares it as the prefix of the 
correct information bitstream. Then, it reinitializes the situation, according to the 
procedure discussed for the reinitialization of the down-to-earth decoding. 

We have seen, in an earlier exercise, that our standard hypothesis for a correct 
decoding (the error bitstream has to be “locally thin” in the sense made precise 
by the estimation e, < $(d* — 1) for t > 0) is not sufficient in order to guarantee 
the existence of a common prefix of the candidates in the final list of the interior 
algorithm. 

We probably need a supplementary hypothesis which clarifies the notion “locally 
thin” in a more restrictive way: We will have to add a condition of “discontinuity” for 
the error bitstream: the inspection window moving over the error bitstream should 
meet regularly “clean” sections — where there are no transmission errors. 

But in practice the situation is much simpler. Since the first step of the Viterbi 
algorithm contains the complete information for the down-to-earth decoding, we can 
use it for a correct default decoding whenever the interior algorithm does not arrive 
at a list of candidates with a common prefix. 


Exercises 


(1) Consider the convolutional code with generator polynomial 


vo (8220) =(1)-()e0(0) 


and the received bitstream 


Vo Vi V2 V3 V4 V5 V6 V7 Vg V9 Vio Vii Vi2 V13 V14 V1i5 


10000000011 0 1 0 0 

0103100311100 1 0 0 0 1 

Our interior algorithm will have six steps. 

(a) Find the first 10 bits agai ---ag of the information bitstream (two rounds 
of the interior algorithm!). 

(b) Determine the corresponding error bitstream. 


(2) Consider the convolutional code defined by 


l+2?+2° 1 0 1 1 

l+ae4+2? 1 1 O\ 2 1] 3 
Gla) = ae gectecae? aN a epee | Gi ag 

l+aote?4+e2? 1 1 1 1 
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as well as the first 24 frames of the received bitstream: 


Vo V1 V2 V3 V4 V5 V6 V7 Vs V9 V10 V11 V12 V13 V14 V15 V16 V17 Vis Vi9 V20 V21 V22 V23 


00001011011 001 1 1:0 1 1 0 0 1 1 =+0 

1021100311001 21 0 0 1 21 0 0 1 21 0 0 1 «21 

1o03i13100311001 1 0 0 1 21 0 0 1 21 0 0 1 +21 

10000000000 0 0 0 0 1 0 00 0 0 0 0 0 
The interior algorithm will have 12 steps. 


(a) Find the first 18 bits agai ...a17 of the information bitstream (you need two 
rounds of the interior algorithm — with a reinitialization). 
(b) Keep track of the transmission errors. 


5 


Data Reduction: Lossy Compression 


In this chapter, we shall treat algorithmic decorrelation methods for digital signals, 
based almost exclusively on the tools of Linear Algebra. 

So, we will be confronted with certain (invertible) linear transformations which 
permit a representation of the numerical data in a form appropriate for decisions of 
the kind “let’s save all important features and let’s delete everything which can be 
neglected”, while maintaining at the same time a strict quantitative control over the 
truncation effect. 

In a more traditional mathematical language: We search for optimal approxi- 
mations of our numerical data units, i.e. for approximations the quality of which is 
defined with the help of linear transformations that “unmask” the underlying digital 
information. In order to enter the subject by a familiar gate, we shall begin with 
a complement on the Discrete Fourier Transform, more precisely on its action in 
trigonometric interpolation. This will permit us to introduce the basic notions of 
digital filtering in an ideal outset “where everything works neatly”. 

But it is not the DFT which is the typical decorrelation transformation in digital 
signal theory. 

The center of our interest during this chapter will be digital image processing. 
This makes the Discrete Cosine Transform (JPEG) and the Discrete Wavelet Trans- 
form (JPEG 2000) appear. 

The Discrete Cosine Transform is a classical real orthogonal transformation of 
signal theory, a kind of real part of the DFT, which has to be appreciated as the 
simplest member of the great family named the Karhunen-Loéve Transform.! Its 2D 
version, acting on matrices, is best understood ad hoc: It preserves the Euclidean 
matrix norm (the “discrete energy”) and diagonalizes constant matrices. As a con- 
sequence, it clusters the numerically significant information of matrices with small 
numerical variation in the left upper corner of the transformed matrix, thus creating 
the desired situation for efficient quantization. The Discrete Wavelet Transform is, 
exactly like the KLT, a whole family of transformations. In their operational ver- 
sions, 2D Discrete Wavelet Transforms act via matrix conjugation on matrix schemes 
— completely similar to the action of the 2D Discrete Cosine Transform. Now, we 
have lost orthogonality, but we have won pictorial significance. More precisely: The 


' There is a surprising uneasiness in literature to associate the Discrete Cosine 
Transform with the KLT — why? 
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matrix versions of the considered Discrete Wavelet Transform describe the action of 
a couple of digital filters — the low-pass filter computing local means, the high-pass 
filter annihilating highly regular progressions. Thus, in 2D version, the transformed 
matrix scheme splits into four sections LL, LH, HL, HH, according to the possible 
horizontally /vertically combined actions of the filters. The section LL consists of 2D 
local means, i.e. it is nothing but a lower resolution (of smaller size) of the initial 
matrix scheme; the other three sections have no real pictorial meaning — they contain 
the numerical details for faithful reconstruction. 

All this is filter bank theory; it can (and will) be explained without reference 
to wavelets. What about the wavelets supporting the whole discrete machinery? 
Roughly speaking, the fact that a two channel filter bank is indeed a Discrete Wavelet 
Transform has an effect on the quality of the reconstruction: after courageous quan- 
tization, we will then be able to keep distortion reasonable. At any rate, compared 
with usual (more or less acoustic) signal theory, we work here in a rather delicate en- 
vironment: The theory of the digital image is an autonomous discrete theory (which 
is not derived from some continuous theory). In principle, the spatial variable should 
not be stressed by the standard “time/frequency” duality (the reader will observe 
that we avoid spectral arguments as long as possible, and use them only in purely 
technical considerations). So, the No signal theory without Fourier transform — par- 
adigm should be seriously relativized. On the other hand, the design criteria for 
our best transformations are precisely imported from these (conceptually) invalid 
domains — and finally yield extremely satisfactory results. 


5.1 DFT, Passband Filtering and Digital Filtering 


Recall The Sampling Theorem for elementary periodic signals can also be stated 
as a trigonometric interpolation theorem in the following way: 


Let fo, fi,.--;fn—1 be n = 2m complex values. 

Then there exists a unique balanced trigonometric polynomial of degree < m 
f(t) = rs + a1 - cos 2nt + bi - sin2nt + az + cosdnt + bz -sindat + +--+  - cos 2amt 
with f(<)=frO<k<n-1. 

Note that in the spirit of this chapter, the Discrete Fourier Transform of order 
n — which associates with the vector of n equidistant sample values the coefficient 
vector for the complex notation of f(t) — indeed carries out a kind of decorrelation: 

The action of F, uncovers the frequential information contained in the list of 
given ( time domain) values. The natural (lossy) compression thus proposed would 
be based on a quantization consisting of the annihilation of certain frequencies. So, 
we enter the domain of (digital) filtering. 

Our first experiences there will be made in the context of the extremely well- 
behaving periodic theory. 

In this section, we shall essentially treat passband (low-pass, high-pass) filtering 
on elementary periodic signals, which will be done as digital filtering on the sequences 
of their sample values. 

Mathematically, we only have to efficiently combine the Convolution Theorem 
with the Sampling Theorem. 
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Filtering by (Circular) Convolution 


Let us first fix the language and the formal setting of our presentation. As often in 
Mathematics, a rich dictionary does not necessarily mean a rich underlying structure. 
Things are still rather simple. 

We shall begin with a 


Definition A discrete n-periodic signal is an (infinite) sequence z = (Zk)rez of 
complex numbers such that Zp4n = Ze for all k € Z. 

Such a sequence is completely determined by the segment (20, 21,..-, 2n—1)- 
Notation z= ||: 20, 21,...,2n-1: |]. 


Example Let z(t) be a (continuous) periodic signal, of period P. 
Sample to the order n (i.e. at n equidistant nodes in any interval of length P). 
We obtain a discrete n-periodic signal z = (zx)kez, where 2, = 2(£P), k € Z. 
Note that a priori there is no relation between the sampling frequency n and the 
period P of x(t). 


Observation The discrete n-periodic signals add, and are multiplied by complex 
scalars, in an obvious way (they are functions defined on Z). 

Let IIn be the (complex) vector space of all discrete n-periodic signals. The 
standard basis of IIn is given by 


eo = ||: 1,0,0,...,0: |}, 

e1 = ||: 0,1,0,...,0: |}, 

€n—-1 = ||: 0,0,0,...,1:]]. 

Hence: z = || : 20,21,---,2n-1 : || € TI[n can be uniquely written as a linear 


combination of thee;, O<i<n-1: 


z= zeo + 21€1 +°++ + 2Zn-1€n-1- 


Observation The (circular) convolution product *: C” x C” — C” extends natu- 
rally to a convolution product *: II, x I, — IL: 


=|): moti, ---52m—1 2 [L,Y =I] 2 YosYs--- Yo : | 
z=a£*y=||: 20,21,---,2n-1: || is given by 
n-1 
ae =) > aye, O0<k<n-1. 
j=0 


Moreover, the cyclic permutation ¢ : C” — C” defined by 


20 2n-1 
21 20 
fon — 
zn-1 Zn—2 


extends to a right-shift transformation on In: 
Ol 3203-Zis eee SRE ||: lee Fossa enee || 


270 5 Data Reduction: Lossy Compression 


If z = || : 20, 21,---,2n-1 : |] represents an (equidistant) sampling of the P- 
periodic signal z(t), then oz = || : 2-1, 20,..-,Zn—2 : || represents the corresponding 


sampling of the signal z(t) = z(t —), with r = +- P 


n 
Now we are ready to state and to prove the principal result of this section. 


Definition An invariant linear filter (acting ) on the discrete n-periodic signals is 
a C-linear transformation 
T:TWn— IIn 


which is insensitive to (discrete) time shifts, in the following sense: 


T(oz) =oT(z)for all z € Th. 


Remark Let h = ||: ho, hi,...,;hn—1 : || be a fixed n-periodic sequence. 
Then the mapping 7}: Wn — In 
given by 
Th(z) = z*h 


is an invariant linear filter. 

Note that h = T),(eo) = eo * h, ie. h is the image of the elementary impulse 
€0 = ||: 1,0,...,0: || by the filter T,: h is its impulse response. 

The following proposition shows that there are no other invariant linear filters. 


Proposition Let T :TWIn — Wn be an invariant linear filter. 
Consider h = T(eo), 
i.e. || : ho, Ai,..-,hn-1: || = T(|| + 1,0,...,0: ||, the impulse response of the 
filter. 
Then we have 
T(a)=a*h forall «€Wn 


(the filtering is done by convolution with h = T(eo)). 


Proof Consider h = T(eo) = || : ho, hi,.--,hn-1 : |l- 
But e1 =9e€0, €2 =07e0,..., €n-1 =o” ep and T(ox) = oT(x); hence 
T(e1) = | = An—1, ho, . o .,An-2 ? ||. 
T(e2) = | 4 An—2,hn-1, et .,An—3 a ||. 
T(e€n-1) = | A hi,he,.. ., ho £ ||. 
For x = || : @0,71,...,%n-1: ||] = voeo + Vier +... + 2n-1€n-1 we obtain: 


T(x) = xoT (eo) +21T(e1) +--+ +an-1T (en-1) = 
xol| : ho, hi, esate An—1 E | 
+21|| : hn-1, ho, bes shn-2 7 | 


Ln—1|| :hi,ha,..., ho : | 


= || S- Bi Neon ye, Lmhi-m,.--, Se titi :||/=axh as claimed. 


Oo 
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Exercises 


(1) Let T : Ig — Ig be the smoothing filter defined by 
T(x) =y = Yr = £0p—-1 + 50K + tae kez 
(a) Show that T is linear and invariant. 
(b) Find the impulse response h = ||: ho, hi,...,h7 : |l- 
(2) Let A:Wn—-TIy be the filter of “discrete differentiation” defined by 
A(z) =y — Yk = Lk — Lk-1 keg 
(a) Show that A is linear and invariant. 
(b) Find the impulse response d= ||: do,di,...,dn—1: |]. 


Digital Filtering of Sampled Periodic Signals 


Situation Consider elementary periodic signals of the form 
x(t) = Y + a1-cos2rt + bi -sin2nt + azg-cosdnt + be-sindat +... S*-cos2amt 
(i.e. balanced trigonometric polynomials of degree < m). 
We are interested in linear filterings 
a(t) > &(t) = So + @-cos2at+b,- sin2at+ Go-cos4nt + bo-sindat+... tye . 
cos 27mt 
which factor through digital filterings in the following sense: 


x(t) a a(t) 
| sampling reconstruction T 
| to the order by trigonometric | 
| n=22m interpolation | 
ro Xo 
U1 digital filtering 1 
: — ‘ 


of the sample values 
En-1 In-1 


Let us look more closely at the situation: 

So, let x(t) = Y + a1-cos2nt + bi - sin2nt + az-cosdnt + ba-sindnt +... S- 
cos27mt be a (balanced) trigonometric polynomial of degree < m. Sample to the 
order n = 2m: 


Xo x(0) 
£1 = «(=) 
n—1 a(2) 
An invariant linear filter on the sample values will act via ( circular) convolution: 
Let h = || : ho,hi,l...,hn-1 : || be the impulse response which defines our 


invariant linear filter; then 
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2 ho x(0) 

: a he z(+) 

a: n-1 
En-1 hn (en) 


&(t) will be the (balanced) trigonometric interpolator of degree < m for the vector 


Ln-1 


We know, by the Sampling Theorem: 


— Com ,—2rimt Qrivt Cm ,2riLmt 
If a(t) = Re + Sere et fuk ge 
~ _ €_m .—2nimt ~ Qrivt Em ,27imt 
and @(t) = —"e ca Do pieneaea Cve + re 
then 
Xo Co Zo Co 
X11 C1 aa C1 
1 : Cm—1 1 . Cc -1 
—F,, = i and —F, =]... 
n : Cm = C-m n : Cm = C-m 
C-m+1 C24 
Ln-1 C-1 In-1 Cad 
Hence: 
Co Ho + co 
C1 Ay +c 
Cat _ Am—1 Cm—-1 
CH= Con Hm + Cm 
C-m+4+1 Am-+1 *C-m+41 
C-1 An—1 + C-1 
Ao ho 
Ay hy 
where =F, 
An-1 An-1 
Once more: 
€o = Ho - co 
C1 = Ay “C1 
C1 = Ayn-1 cick Clea 
Let us sum up: Let h = || : ho, hi,...,Rn—1 : || be the impulse response of the 
digital filter acting on the sample values of our elementary periodic signals 
(n = 2m, where m is the maximum degree of the considered trigonometric 


polynomials). 
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Then the analog filter defined by 

sampling —> digital filtering— reconstruction (of the analog nature) by trigono- 
metric interpolation 

is a “multiplicator on the complex amplitudes” . 


Ho ho 
More precisely: Let me =F, @ 
An-1 An-1 


Then 


# _ €_m _—2rimt ro ~ a2mivt | Em .2rimt 
Z(t) = 7e ae ee Cve 2 © 


is obtained from 


x(t) = aoe eee dea a Cue 
by means of simple multiplications on the coefficients: 


2mivt | Cm e2timt 


In particular: Every passband (low-pass, high-pass) filter is obtained this way: 
Put H, = Hn-, = 0 for the frequencies v you would like to annihilate, and 
A, = Hyn-v =1_ for the frequencies v you would like to preserve. 


Exercises 


(1) Consider the smoothing filter T:IIg >IIg defined by 
T(| 2 U0,%1,...27 : ||) = | > Yo, Y1,---Y7: ||) with Yk = $Up-1 + ta + 
t0b41 keZ. 
Following the procedure described above, we shall construct a linear filter on 
the elementary periodic signals (more precisely: on the balanced trigonometric 
polynomials of degree < 4) in the following way: 
Sampling to the order 8, digital filtering (by T) of the sample values, recon- 
struction (of the analog situation) via trigonometric interpolation. 
A signal 
x(t) = BY + a1-cos2nt + bi - sin2at + ag-cosdrt + be- sindnt + a3-cos6rt + 
b3 - sin6nt + S - cos8rt will be transformed into 
Z(t) = au + & - cos2nt + by - sin2xt + Gg - cosdrt + be - sindt + Gg - cos6mt + bs - 
sin6at + a4 - cos87t. 


(a) Write the coefficients of ¢(¢) in function of the coefficients of x(t). 
(b) What happens if you iterate the operation of T? 

(2) We consider the elementary signals of the form 
x(t) = 2 + ay-cos2rt + by: sin2at + az-cos4nt + by: sindnt + a3 -cos6at + 
b3 - sin6nt + 4 - cos8rt. 
Construct a low-pass filter by means of a digital filter: Sampling to the order 8, 
digital filtering of the discrete 8-periodic signal, trigonometric interpolation of 
the output values. 
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The result of the filtering of 

x(t) = BY + a1-cos2at + bi - sin2at + ag-cosdrt + be- sindat + a3-cos6rt + 
b3 - sin6rt + S - cos8rt 

should be 7 7 7 
&(t) = BY + G1 - cos2at + bi - sin2at + Gz - cosdrt + bz - sindrt + G3 - cos6rt + bs - 
sin67t + 4 -cos8rt = “ +ay,-cos2rt + 6b; - sin27t + a2 - cos4mt + bo - sindzt. 


(a) Find the impulse response h = || : ho, hi,...,h7 : || of the digital filter to 
be constructed. 
(b) A little test: Sample x(t) =cos6mt to the order 8, filter the sample values 
by T;, and verify that the result is indeed the zero vector. 
(3) Consider the elementary signals of the form 
a(t) = 2 + a ,-cos2rt + b1-sin2rt + a2-cos4rt + be: sindrt + a3-cos6mt + 
b3 - sin6at + $ - cos8rt. 
A channel has the undesirable effect to weaken the high frequencies. We shall 
model it as a linear filter which associates with x(t) the signal 
y(t) = on +ay,-cos2zt + bi -sin2a¢ + = -cos47t + % -sindat + -_ - cos67t + as : 
sin6mt + $$ - cos8rt. 
We would like to reconstruct the initial signal by means of a digital filter: Sam- 
pling of y(t) to the order 8, digital filtering of the discrete 8-periodic signal, 
reconstruction of the signal x(t) via trigonometric interpolation. 
Compute the impulse response h = || : ho, hi,...,h7 : || of the digital filter to 
be constructed. 
(4) We consider the elementary periodic signals of the form 
x(t) = 2 + aycos2nt + bisin2nt + agcos4nt + besindat + agcos6nt + bgsin6at + 
% cos8nt. 
Construct a (high-pass with phase shift) filter 
a(t) + &(t) = agcos4r(t — £) + besinda(t — ¢) + ascos6m(t — £) + bgsin6m(t — 
+) + Scos8x(t — 3) 
by means of a digital filter: Sampling to the order 8, filtering of the sample 
values, trigonometric interpolation. 
Find h, the impulse response of the digital filter. Which is the vector H of the 
multiplicators on the coefficients (of the complex notation) of the considered 


signals? 


5.2 The Discrete Cosine Transform 


In this section we shall treat the transformation which is perhaps the most popular in 
the big family of the orthogonal transformations which serve in signal theory. On the 
one hand, the Discrete Cosine Transform (DCT) is a kind of correctly defined real 
part of the Discrete Fourier Transform, on the other hand, it is an (almost trivial) 
member of the family of the Karhunen—Loéve transformations, covering the case 
of highest correlation. Under this aspect, it will often be taken as the predefinable 
substitute for the other members of this family, which are algorithmically almost 
inaccessible, at least in real time problems. 

As it has already been pointed out, the 2D version of the DCT — acting in 
digital image compression — is perhaps best appreciated without reference to its 
1D constituent: It simply preserves the Euclidean matrix norm and diagonalizes 
constant matrices — two basic properties which, conjointly, are sufficient to explain 
its great interest for decorrelation problems in image compression. 
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5.2.1 Functional Description of the DCT 
In a first approach, we shall be concerned with the properties of the DCT which 
stem from its connection with the Discrete Fourier Transform. The more conceptual 


properties which explain the frequent appearance of the DCT in engineering will be 
treated in the later sections. 


Definition and Fundamental Properties of the DCT 


Definition The Discrete Cosine Transform of order n, 


R” — R”, 


is defined by 


So pat 
Xo= Fa Du0<k<n—1 Tk 
on 2 J2k+1) ni 
Xj = V2 Vo cnen_1 te cos Nr 1l<j<n-l. 


In matrix notation: 


me (Qk +1) 
CG. [20 cos T)O<j,k<n—1 


al ;-0 
win fe 


j =the row index 
k = the column index 
Example 
1 1 1 1 
2 2 2 2 
1 cos= cos2= —cos® —cos2 
C 8 8 8 8 
4 = Kin us as wT 
2 | cos} —cos% —cos% cos] 
3a us us 30 
cos —cosg COsZ cosy 
Proposition C, : R” —_ R" is an orthogonal transformation, i.e. the 


matrix C', is invertible, and C+ = Ct, (the inverse matrix is equal to the transposed 
matrix). More explicitly, the inverse DCT is given by the formulas 


a 2 (27 + 1)k . 
_ 7 y néogsuee <j<n-1. 
Xj Um ‘2 Xk + COS = nm O<7<n-1 


1<k<n-1 


Proof This is the theme of the following section “DCT and DFT”. 
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Remarks (1) The most natural matrix for the DCT of order n would be the 
matrix 
oa Ca | 
an 0<j,k<n—1 


The normalizing coefficients fe “Tj; | are necessary in order to correctly 


obtain CyaCe = In = the unit matrix of order n. 
200...0 
010...0 


Actually, we have Cy; -Cy' = 2 001...0 
000...1 
(2) The orthogonality of the matrix C, can also be expressed in the following way: 


The columns of C;, are a orthonormal basis of the R”. 
The argument is simple: We have, for every matrix A € R"*”: 


<¢€1,C1 > < 1,02 >... <C1,0n > 


<€2,C1 > <€2,C2 >... <C2,0n > 
At-A= 


<€n,C1 > < Cn, C2 >... < Cn, Cn > 


i.e. the product of the transposed matrix with the considered matrix is equal to 
the matrix of the inner products of the columns c;, 1 < j <n, of the matrix A. 
We shall return to this aspect in a while. 


Exercises 
1000 
1, {0100 
(1) Verify that Cs-Cy =I, = 0010 
0001 


(2) Write down explicitly the matrix Cs € R®°*®, taking in account the following 
numerical values: 


Cc, = cos 16 = 0.9808 c2 = cos g = 0.9239 c3 = cos # = 0.8315 c4 = cos - = 0.7071 


C5 = COs ar = 0.5556 cg = cos ot = 0.3827 c7 = cos _ = 0.1951 
Answer 
1 1 1 1 1 1 1 1 
V2 V2 V2 V2 V2 V2 V2 V2 
cos 47 cos £7 cos 27 cos <n cos 7 cost cos 2x cos 2x 
6 6 6 6 6 6 6 16 
2 6 ) 4 8 22 26 30 
COS 7g7 COS 7g7 COS T_T COS FET COS TET COS T_T COS TET COS TET 
fe 5 < ‘ a 
1 cos 2 cos 27 cos 27 cos 247 cos 227 cos 227 cos 227 cos 27 
oe 6 6 6 6 6 6 6 16 
2 4 2 20 28 36 44 52 60 
COS 7g 7 COS 7_M™ COS Fg COS TEM COS FET COS TZN COS [ET COS TET 
5 5 25 35 45 55 65 75 
COS 7g 7 COS 7gT COS TET COS TEM COS TBM COS TZg7 COS 7gT COS TET 
6 8 30 42 54 66 78 90 
COS 7g 7 COS TEM COS FET COS TZM COS TET COS TEM COS T_T COS TET 
7 21 35 49 63 17 91 105 
COS 7g 7 COS TZN COS FET COS TZM COS TET COS T_T COS FET COS 4g 
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C4 C4 C4 C4 C4 C4 C4 C4 
C1 C3 C5 Cy —C7 —C5 —C3 —C1 
C2 C6 —CE6 —C2 —C2 —CE CE C2 
1 C3 —C7 —C1 —C5 C5) C1 OC? 3 
2 | c4—ca —Ca Ca Ca —Ca —Ca C4 
Ca —C, Cry C3 —C3 —C7 Cy —C5 
ce —C2 C2 C6 —Ce C2 C2 C6 
C7 —C5 C3 cy Cy C3. «5 C7 


DCT and DFT 


First observation: 

Consider Fy =(wi*\o<j nent; the matrix of the DFT of order n. Its real 
part Re Fy, =(cos 228 )ocj ncn—t is, unfortunately, a very bad candidate for an 
orthogonal transformation: For n > 3, Re F,, is not invertible. 


Exercise 


(a) Show that Re F¥ is not invertible. 
(b) Generalize: Show that Re F;, is not invertible, for n > 3. 


We shall see that, up to a certain (modest) algebraic deformation, the DCT is 
the correct “real part” of the DFT. 

Choose n = 2m even. 

First two auziliary definitions: 


a0) 
1 
(i) Put, for «= . € R" 
Ln-1 
oo) 
~, v2 
Lo Xo 
Zi X1 
Cio 
z= = ical (ey Os 
En-1 
Lid In-1 
x3 
Ly 
(with the obvious permutation matrix P,, € R”*”). 
Explicitly: 


Lj = 2; : 
5 eee 0<j<m-1 
Ln—j-1 = C2j41 


(ii) Let Qa, €C"*" be the following diagonal matrix: 


10 0... 0 
Owan 0 0 
2 
Qn = 0 0 Wins. 0 (with wan = cos = — isin =). 
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Definition Ty = QanFnPp € C”*”. 


Proposition Re T,, = (cos TORT) Jo<i,e<n—1 
(j = row index, k= column index). 


0) XO 
21 Ly 
Proof Write ’ =Th 
Zn—-1 Tn-1 
i.e. 
eer) Bc Ee ; 
25 = Win ock<n—1 LkWA O0<j<n-1 
Hence: 
a jk j(n-1-k 
25 = Way, * ( S LW + s tan+1w) ) 
0<k<m-1 0<k<m-1 
we jk J —j,,—Jjk 
= Wan LS L2KWy, + Wan * x L2k41Wyn Wy 
O0<k<m-1 0<k<m-1 
a j(4k+1) —j(4k+3) 
= ) L2kW4), + T2k+1W gy 

0<k<m-1 0<k<m-1 

We obtain: 
oes : j(4k+1) ; j(Ak+3) 
Re z; = PS eieeack Lop cos + gees: Ton41 ° cosS a = 
. G(2k+1) . =. 
oeeeask Uk + COS a, Usgen>4 


which gives our claim. 


Complementary observation: With the notations of the proposition, we have: 
Re zn-j =—-Imz; 1<j<n-1. 


a0) 
T1 
Proof x= : € R”, =Fi(t) > yn-j=J9; l<j<n-l. 
Ln-1 
On the other hand,  wi7? = —iw;? (since wy, = —i). 
Hence: zn—; = —tZ; l<j<n-1. 
Then: Re zn—j = 4$(2n—j + 2m—j) = 3(-i2j + izy) = $(2;) — 2) = —Im 2; 


eee ek 


Algorithmic consequence: 


Fast computation of X* = CF (x) for r= . €R", n=2° 


Ln-1 


(2k +1 
C= (os ioe) j = row index k=column index 
Ct 0<j,k<n-1 


(1) Passage of x to & = P,(x) ( permutation of the positions). 
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(2) Computation of y = F,(@) by FFT. ; 
(3) Multiplication of the components y; of y by the wi,,0 <j < n—1. We 


obtain z. 
ZO 
Rez, 
(4) Output X* = Rezm i SS Sy 
—IMZm_1 
—Imz 


We insist: The vector x and the vector X* are real. 


Exercise 


Compute X* = Cg(«) for 


and for x= 


Nl] Re 
PrRFoOoNoOrFcorF 


SNOW WNEH 


At the end of this section, we shall compute the matrix C,*, as promised. 


Lemma Ty = Q4nFnPn is invertible, and T71 = tie 


(the matrix Te is the transposed conjugate matrix of the matrix Ty). 


Proof We have: 2; = Qan (inversion of the roots of unity on the diagonal) 
and E+ = Fh and P,* = P£ (for every permutation matrix P € R"*" we 


have P~' = P*), 
Hence: Tz? = PytF,10;) = 1 PEF Quan = 2 PLFA Qin = 4(QanFnPn) 


t 


10...0 pee 
00...1 
00...0 0 10 
Notation i=]... ‘ ER a a € R"*" 
00...0 01...0 
Recall J, = +Fy (a previous exercise). 


Lemma T,,Ti + TaTt = 2nly 
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10... 0 
00...-i 
Proof T,T’ = Qan Fn Pa Ps Fy in = Day FA Qin = nO Ia Qin = 
: 0 
0O-i... O 
2n 0 ...0 
0... 0 
Hence: T,T% + TnTt = 2nty1 = : 
Ob ve nceus 0 


Proposition Let Cy, =Re Tn. 


oow 
oro 
a) 

oo 


Then o*cxt = 5+ 1) = a 


Ores vas. cd 


Pinch Ce SAT, FT) Cr a= GGT) OACe = MT eT 


Tie va ly Chee eT 
$(2nlir oe 2n1) = $U + Ih1) 


Corollary Let 


Io = (1,1,...,1) 
1, = (cos Beye COS Grae) 
lp = (cos 22,..., cos 2Gn— UF) 
In-1 = (cos eee .., COS in tuee Ln) 
be the n rows of Ch. 
Iolé =n 
Then 1lj = 2 forl<j<n-1 
Ili =0 forj #k. 
This yields: 
ly 
Vn 
2 
Cn = is an orthogonal matrix: C;' = Ch, 


and proves the inversion formulas of the earlier section. 


(1) 
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Exercises 
Xo 
Xo ‘ 
: . Ly : : “ x 
Associate with «= €R* its “mirror twin? #= 3] eR’. 

x2 x3 

x3 
na0) 


Put Fu(ax) = Fe(4). ; 
(a) Write down the matrix F, € C8**. 
(b) Show that 


1 
4,000 
— 1 {0100 a 
Ga |g gi. 8 2 
0001 


(c) Generalize (replace 4 by arbitrary n). 

Find the identity which compares, for x,y € R*, Ca(a * y) and C4(a) - Ca(y) 
(where the product in the first term is the circular convolution product, and the 
product of the second term is meant component by component). 

Generalize (replace 4 by arbitrary n). 

In the sequel, we shall denote by X; the kth component of the vector C,,(x), 
where n = 2,4,8,.... 


Notation ee = cos am 3} = sin a 
Put a= 5 B=ch, 6=sh and o=cle, e=c¥g, w= 53a, T= Sie. 
111 éii1 Oo pf TE 
_ [| a-a a —-a ~ [wrt -€o 
Let Ty = Cue eos and D4z= ee ae 
) —$6 — T€ O LL 
Xo Xo 
Xo _F x2 
(a) Show that peal aes eer 
X3 ry 
Xo XO 
X4 x2 
X2 3 LA 
‘e X6 TT T4 x6 
b) Verify that =[ x a 
( ) ~ X41 (4 4) x7 
X5 x5 
X3 X3 
X7 Uy 


Let Poi be the matrix of the Discrete Cosine Transform to the order 2” in 
permuted version: The input vector will be in “increasing even positions — de- 
creasing odd positions” ordering, and the output vector will be in bit-inverted 
enumeration; we deduce that 
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™ 7% 
Tg = 
Da —Da 
(4) Put y, = 4%" for k= 0,1,2,3. 


(a) Write T, in function of the cos jpr, j even, O<k <3. 

(b) Write D4 in function of the cos jpr, j odd,O<k <3. 

(c) Using the identities cos(2m + 1)y = 2cos(2m)yp - cosy — cos(2m — 1)y, you 
should obtain 


COs Yo 
Ret cos Yo - (1 + 2cos4yo — 2.cos 20) 
oor cos yo - (—1 + 2cos 2y0) nae 
cos Yo - (—1 — 2cos4y0 + 2.cos 2y0 + 2cos 6yo) ... 


(d) Verify the following equality: Dy = P{LaP{TiQ4 


with 
1000 
" 0010 
fe ono: (i 
0001 
1 0 00 
-12 00 
Ia= | 1 9 20]? 
—-1 2 -22 
cosyo 0 0 0 
= 0 cosy: 0 0 
Qa= 0 cosye 0O 


0 
0 0 0 cos y3 
(e) Generalize (x oe Tact): 


Orthogonal Matrices and Hermitian Matrices 


Consider first the R” (i.e. the vector space of real n-tuples, in column notation), 
with its Euclidean structure defined by the inner product 


xo Yo 
(x,y) =a-y =aoyoteiyit::tan—-1yn-1 for w= ome eae 


In-1 Yn-1 


The Euclidean norm of the R": || @ ||= /(x,2) = fei + az t+---+02_, 
If the vector x has a geometrical meaning, we shall speak of length; if the vector 
x is the list of the sample values of a signal, we shall speak of energy. 


Recall We have, for every matrix A € R”*” and every couple z,y € R”: 
(Az, y) = (x, A’y) 

This elementary identity is the basic tool for every transfer between the arith- 
metical properties and the geometrical properties of the matrices. 
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First, we shall deal with orthogonal matrices: 
For a matrix Q € R"*” the following properties are equivalent: 


(1) Q is an orthogonal matrix, i.e. Q is invertible, and Q~' = Q¢ (the inverse matrix 
is the transposed matrix). 

(2) || Qa |/=|| x || for all « € R®: 
the linear transformation “multiplication by Q” preserves the norm (the length, 
the energy). 


(3) The columns of Q = (co,¢1,...,€n—-1) are an orthonormal basis of the R”: 
_ jl fori =j. 
ceo) = " else. 


Exercise Prove the equivalence of these three statements. 


The generalization of the orthogonality property from the real domain to the 
complex domain needs some care. 

First, for the C”, the vector space of columns with n complex components, a 
C-bilinear inner product (obtained by simple matrix multiplication of a row with a 
column) is not natural, since the definition of a real norm — in formal analogy with 
the case of the R” — does not work. 

Hence we are obliged to accept the following definition: 


20 Wo 
41 W1 
(z,w) = 20Wo + 2101 +--+ + 2n-1Wn-1 forz = : , w= ; EC”. 
Zn—1 Wn-1 
(note that we have lost bilinearity and symmetry: 
(z, \w) = A(z, w), (w, z) = (z,w) !). But we thus obtain the right norm: 


ll 2 l= (2.2) = fae + ye tad + ye t...a2_ +921 


20 Xo + iyo 
Zl ri + iy1 
for z= F = ) 
Zn—-1 Tn—-1 + iyn-1 
identifying the C” with the R?”, in a natural way. 
Now we have, for A € C"™*”: (Az, w) = (z,A'w) for all z,w € C”. 


The natural generalization of the orthogonality in the domain of real matrices 
is the notion of a Hermitian matrix in the complex domain: 
For a matrix Q € C”*” the following properties are equivalent: 


(1) Q is a Hermitian matrix, i.e. Q is invertible, and Q~' = Q (the inverse matrix 
is the transposed conjugate matrix). 

(2) || Qz |l=|| z || for all z eC”: 
The linear transformation “multiplication by Q” preserves the norm (the length, 
the energy). 


(3) The columns of Q = (co, ¢1,...,€n—1) are an orthonormal basis of the C”. 
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Example Until now, we have met only two types of Hermitian matrices: 


1 (jk 
(a) Fr = Sa (wht ) 
(note that we need the factor Fa in order to get a symmetric situation: We 


thus obtain Fz? = Sa (wai*)= F,,; on the other hand, this factor is highly 
unpleasant on account of the recursion formulas for the algorithm FFT — this 
explains its appearance at this late stage of the discussion). 

(b) Tr = QanFnPn with the matrix F,, above. 
Let us point out, once more, that the DCT matrices are orthogonal: 


. + forj=0 
(c) Ch = ies (1; . cosi@kth ) with r;= { Ng rik 


Then C,1=C!%. 


Exercises 


(1) Show the orthogonality of the following two matrices: 


(free? 
Q= 5 ( 22-1 Ee Rex, 
4 <9 


TOR (ess eee 

ae 1-1-1 1 4x4 

Q@=sli1ii1|/¢® 
iT a a 


(2) Let n be even, and consider the matrix Q = (qjx)o<j,h<n—-1 € R”™*”, defined 
by 


4/2 »tos tee for 7 = 0,2,...n—4, 
Jf? -sin Grier for 7 = 1,3,...n —3, 


dk=) 1, ee 
Yr coskr one 2, 
Fi forj =n-1, 


Show that Q is an orthogonal matrix. 
(3) The Walsh-Hadamard transform. 


N = 2" 
Associate with the binary notation of 7: 0 < 7 < N —1 the vector 7 € Fe: 
Ora 
j =Bn-12""! +...12+ 80 will be identified with F=|] : | eFE. 
Bo 


) 
j,k) )o<inen—1 € RN*N 
1), H(2) and A(3). 
H(2) H(2) 
) and H(3)= ee an) 


Show that H(n+1)= GS .) forn > 1. 


Let us define: hn(j,k) = (—1)' 
We obtain the matrix H(n) = 


——~ 
x ol 
aK = 


(a) Write explicitly the matrices H( 
( 


(b) We observe: H(2) = ee H 
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(c) Show that HA(n)H(n)=N- In. 
The matrix —LA(n) =27>2H(n)_ is then a (symmetric) orthogonal matrix, 
for alln > 1. 

(d) Let M ¢ F§*? _ be the matrix the rows of which are the 8 binary notations of 
0 = 000, 1 = 001, ..., 7 = 111. 
Compute MM' € F3*°. 
What is the relation of MM" to the matrix H(3) € R°*® ? 


2 


The Fundamental Property of the DCT 
Recall: Diagonalization of Real Symmetric Matrices 


Let A € R”*" be a symmetric matrix (A = A’). 

Then there exists an orthonormal basis {qo,qi,..-Qn—1} of the R” and 
Xo, A1,---An—1 € R_ (not necessarily distinct) with 

Aqo = odo 

Aqi = A141 


Aqn-1 = An-14n-1- 


Language {qo,qi,-.-Qn—1} is an orthonormal basis of eigenvectors of A, cor- 
responding to the eigenvalues Xo, A1,...An-1- 


Convention In the sequel, we shall always suppose Ao > Ai > ++: > An-1- 
(Ao will thus be the maximum eigenvalue...). 


Matrix version of the statement above: 
Let A € R”*” be a symmetric matrix; then there exists an orthogonal matrix 
Q E R°*” 


Ao vee © 0 

0 A... O 
and a diagonal matrix D = 

: - 0 

0... O An-1 


with A = ODO! i&, D=O'A0: 

The argument: Let Q = (qo, q1,---Qn—1) € R”*” (column notation) 

Then: A = QDQ' AQ =QD Aqo = Aoqo, Aqi = A1q1,..- 
Aqn-1 = An-14n-1- 


Important Remark The computation of the eigenvalues Ao, A1,...An-1 of a 
real symmetric matrix A demands generally a certain amount of hard work in numer- 
ical (matrix) analysis (iterative methods). Here we have an essentially non-algebraic 
problem. 

If we know the eigenvalues Xo, A1,---An—1 of A, then the computation of 
Q = (q0, 41,---Qn—1) € R"*” is a purely algebraic problem: 

Resolution of the linear systems Ax = A;z, O <i < n-—1, and orthonormal- 
ization of the “fundamental solutions”. 
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Fundamental Property of the DCT 


te veel 
Tete 

A=]...  .|[eR™ 
i eee 


C, =the matrix of the DCT of order n 
Then: 


J/n0...0 nod...0 
J/n0...0 00 
ro See 
: 0 0 
J/n0...0 0 0 
Hence: 
nQ...0 
00 
A=C,| . 2 Gh 
i 0: 
0 ...0 
with (clearly) Ay=n, Ay =AQg=--: =An-1 =0. 


The columns of C%, are an orthonormal basis of the R”, composed of eigenvectors 


for A. 


Exercises 


(1) Let Cs € R°*® the matrix of the DCT to the order 8. 


80---0 
00---0 
Compute the (symmetric) matrix B=Cg] . C8. 
0 0 
(2) Consider the matrix 
1 1 1 1 1 1 1 1 


2V2 2V2 2V2 2V2 2V2 2V2 272 272 
ve 2 1 1 1 1 1 1 


2/14 2/14 2/14 2/14 2/14 2/14 2/14 2/14 

0 6 1 1 1 1 1 1 

42 42 42 42 42 42 42 

0 0 5 1 1 1 1 1 

Qs = 30 30 30 30 30 30 
0 0 0 2/5 2 5 2¥5 2¥5 2¥5 

0 0 0 0 2V3 2Vv3 2/3 273 

0 0 0 0 0 0 “51 s/o: 


Show that it is orthogonal and that it diagonalizes the same matrix A = 
[eet 
as the DCT Cs. 
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lo 
(3) Let A = (co,...€n-1),B = : € R™*“" (notation in n-tuples of 


columns and of rows). 
Show that A -B= Colo t ch ees Cn—11n-1. 
(Every cil; € R"*” O0<i<n-1 !). 
(4) Let Q = (qo, q1,°+:GQn—1) € R”*” be an orthogonal matrix, and let 


No ee 0 
OMe 0 
D=]|. . € R"*” be a diagonal matrix. 
: *. 0 
(eee aes eee 


Show that QDQ* = Aogogg + Arqiay +--+: + An—1Gn—14/,_1. 


Recall The matrix qoqj € R”*” is the matrix of the orthogonal projection 
on the straight line D = Rqo. 


0 Je 0 O 
=z 0 5 0 
(5) Consider the matrix S = a 1 : 1 | €R***. 
2 1 V2 
0 0 ei 0 


Let T,(x) =cos(n@), x =cos0, be the nth Tchebyshev polynomial, n> 0. 
Put 2x, = cos*® k =0,1,2,3. 


Show that Svp = LEVE 0<k <3. 
Is {vo, V1, V2, v3} an orthonormal basis of the R* ? 


Complements on the Eigenvalues of the Real Symmetric Matrices 


Approximation of the Eigenvalues 


In order to insist on the extra-algebraic invasion in the diagonalization procedure 
of a real symmetric matrix, we shall give an example of an iterative method which 
determines the eigenvalues of a matrix A = A‘: The Jacobi algorithm (1846) for the 
diagonalization of a real symmetric matrix A € R””*”: 

Basic idea: Construct a sequence of similar matrices, by conjugation with cer- 
tain simple rotation matrices which annihilate, at every step, the predominant off- 
diagonal couple in the current matrix (fly swatter principle). 

The iteration: Ag =A, Ar = Ry Ari Rij, kee, 

where the rotation matrix R,z is determined as follows: 

One aims at annihilating the off-diagonal coefficient of maximum absolute value 
in the (symmetric) matrix Ax-1. 

| Note that during the iterations the position may again become non-zero, but 
will be of lesser importance...]. 
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Let then aor be the critical element of the matrix A,_ (sometimes, you will 
have to make a choice). 


Rx will correspond to a rotation in the plane (p,q), and the angle @ is chosen 
such as to annihilate a 


Too TOL +++) TOn-1 
T10 Til ++. Tiyn-1 
More precisely (suppose p< q): Ry = 
Tn-1,0 Tn—-1,1 +++ Tn-1,n-1 
with 


Tpp = Tqq = CosO 
Tpq = —qp = Sin 8 
rao =1 i#p,gq 
rij =O else 
Note that the matrix A; differs from the matrix A,_1 only at positions concern- 
ing the rows and columns of indices p and q. 


The modified values will then be: 


(kK) _ (k) _ (k-1) 
Aj, = Ap; = a; 


(k-1) | 
a;,  cos@+a;, sind 


tF#D.q 
(k) _ i(k) _ _(k-1) | (k-1) 
Aig = 44; =A, “sind+a;, ’cosé 
as) = ak) cos? 6 + 2alk-)) cos @ sin 6 + afk) sin? 0 
al’) = ak} sin? 6 — 2ak-) cos 0 sin 8 + al,» cos? 6 


a) = al®) = (ak-» ase) cos 6 sin 8 + ae (cos? 6 — sin? 6) 
In order to obtain al®) = ast) = 0, we have to choose @ this way: 
atk?) : T 
tan 20 = “he=1) __(e=1) with | 0 |< 1 
PP a4 


(If aes = ao one chooses 6 = 4 


z, where the sign is the sign of afk), 
The convergence: 


for k-> om. 


. Neat 


Note that Ao, A1,--.-An—1 are not only the eigenvalues of the matrix A, but also 
those of every “intermediate” symmetric matrix Ax. 


k 
ay Oe sxc 
0 af®) 
Write now A, = Qi, +--+ | + Ex, 


where F;, coincides with the matrix A,, except on the diagonal where it is 
identically zero. 


Let us show that ||£,|| —> 0 for k — oo. 


||Ex|| = the Euclidean norm of Ey as a vector of length n?. 
We have: 


parC she oO) = Silla +4 (al*-D)2) 


iq 


a 
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Hence: 

||Zxl|? = Beall? - 2¢e85-? YP (ap) = aly’ = 0). 
But | ae? | is maximum for E,_1: 

||Zell? < (1 — ste )ilEe-1ll? < (1 - ate) "|| Zoll?. 


This gives the convergence, as claimed. 


Exercises 
(1) Let 


A=[{ 0 1 0 
-iV30 3 


(a) Find, following the Jacobi algorithm (which reduces here to its first step), 
the eigenvalues Ao > Ai > A2soof A. 

(b) Give an orthonormal basis {qo, qi, q2} of the R®, composed of eigenvectors 
for A, corresponding to the eigenvalues Ao > A1 > Ag «of A. 


(2) Let 
3 “Leet 1 
AONE tee eek 
1 1-13 


(a) Beginning with A = Ao, compute Az, the second iterate of the Jacobi algorithm 
towards the diagonalization of A. 

(b) Find an orthonormal basis {qo, qi, q2,q3} of the R*, composed of eigenvectors 
for A. (We observe: \ = 2 is clearly an eigenvalue of A). 


Extremal Properties of Eigenvalues 


Let A € R"*” be a symmetric matrix. 
Consider the quadric qa(x) = 2’ Ax = is Aig LiL; 
Xo 
for «= € R", with A =(4aij )o<i.j<n—1- 


Ln-1 
Let Ao > Ai >--: > An-1_ be the eigenvalues of A, and let {qo,qi,..-Gn-1} 
be an orthonormal basis of eigenvectors for (the multiplication by) A. 
We are interested in extremal values of qa(x) on the unit sphere 
S°-) = {a ER”: ||a|| = 1}. 
The first important result of this section is the following: 


Proposition The situation as above. 


Then: max{qa(x) : ||z|| = 1} = Ao, and this maximum is attained for x = qo. 
Impose now orthogonality constraints: 
max{qa(z) : ||z|| = 1 and (2,qo) =O} =A1, and this maximum is attained for 


xr=q1. 
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In general, we have, forO<s<n—2: 
max{qa(x) : ||| = 1 and (x, Go) = (x, Q1) SSP iiaye SS) (©, Qs) = Oo} = As+1; and 
this maximum is attained for x = Qs+1- 


Proof Let Q = (qo, q1,---Qn—1) € R”*” be the orthogonal matrix the columns of 
which constitute an orthonormal basis of eigenvectors for A. 


Xo Laie 6 
p=|. = Q'AQ. 
: *. 0 
O.--- 0 An—1 


Put y= Q‘x (hence x = Qy). 
We know: ||z|| = 1 <== |ly|| = 1 ( orthogonality of Q). 


Yo 
We have, with « = Qy and y = 
Yn-1 
ga(z) = 2*Ax = (Qy)'A(Qy) = y'Q'AQy = y'Dy = roys + Ary + °° 4 
An-1Yn-1 
Hence: max{qa(z) : ||z|| = 1} = max(Se ye: ce ye = 1} =o 
since ~ MY? < ro ee ys < Xo 


1 
0 
and the value Ao is attained for y = e9 = . |. But: Qeo = qo, i.e. our 
0 
maximum is attained for 7 = qo. 
Now consider the supplementary constraint (z,qo) = 0 (orthogonality 


to qo). 
It means simply that (y,eo) =0, i.e. that yo =0. 
This implies: , 

n-1 n— 
max{qa(x) : |lal| = 1 and (x,qo) = 0} = max{)7 Awl: we = 
1 and yo =0} = max{S-"" NY? : eo. ye =1y=m 
(the same argument as before). 

0 


or 


This maximum is attained for y = e, = ; now, Qe: = qi, hence the 


0 
maximum is attained for 7 = q:. 
The general case is proved in the same manner. 


In order to obtain stronger results on extremal properties of the eigenvalues, we 
shall be obliged to consider a whole variety of real symmetric matrices associated 
with our matrix A € R"*”. 


Definition 1<m<n. Let Um € R”*™ be a matrix of m columns uo, U1,...Um-1, 
which are an orthonormal basis of L(Um) = ke Ru;. 


The matrix Ut, AUm 
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€R™”*™ will be called a section of A. 
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Goo Go1 G02 G03 Go4 100 
10 G11 G12 413 G14 000 
Example A= | a9 G21 22 423 doa | ,€ R°*? U3= | 001], 
430 431 432 433 434 000 
G40 G41 A42 A43 G44 010 
G00 G04 A02 
U3 AU3 = (‘i G44 v:) e R?*?. 
A420 G24 a22 


Lemma Let U},AUm be a section of A. 
Then the eigenvalues of U},AUm are contained in the interval I = [An—1, Ao]. 


Proof Put B = U;},AUm. 
One verifies easily: Ta = cae for all z € R™ — {0}. 
(note that (the multiplication by) Um preserves the lengths, since 


U;,Um = 
Im) 


But: The set {4% : 2 € R” 


el? {O}} is equal to the interval IT = [An-1, Ao] 
(Exercise). 
In applying this auxiliary result as well to the matrix B as to the matrix A, we 


obtain the claim of our lemma. 


The principal result of this section is the following proposition: 


Proposition Let Um = (uo, Ui,...,Um—1) € R”*™ be a matrix of m orthonormal 
columns. Then: min{qa(x) : a € L(Um) NS" "3 < Am-1 

Make now vary Um (but keep m > 1 fixed); then 

maxy,, min{ga(x): 2 € L(Um)NS"~-'} = Am-1 


Proof Write A = QDQ‘, with Q = (Qm,Qm) and Qm = (qo, q1,-+-;Qm-—1): 

Then the second claim of our proposition is — modulo the first one — easy to 
accept, since min{ga(x): 2 € L(Qm)N 8S" '} = Am-1 
As to the first claim, we have to show: There exists vo € L(Um) — {0} such that 


qA(xo) 
‘Teo <Am-1. 


Now consider the linear system Umu = 0Qm-—1 + Qinv. Note that we deal with 
n equations in n + 1 unknowns: u € R™,v € R"”-™,0 € R. So, let (uo, 90, v0) be a 
non-trivial solution. 
x0 =Umuo has to be non-zero, since qm—1 ¢ L(Qin). 
Write 20 = Am—1Qm—1 + AmGm +... Qn—-1qGn-1- 
This gives: 


2 2 2 
Am—1Am—1 + AmAm +:++ + an-1An-1 
2 2 2 
Amn—1 + Am bes Oy 


qA(Zo) _ 
I|x0||? 


< nat 


and proves our proposition. 
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Exercise 


Show similarly: 
max{qa(a) :@ € L(Um)NS"1} > An—m 
and miny,, max{qa(x): 2 € L(Um) NS" '} = An—m.- 


(Help: Write Q = (Qn—m; Qh—m), with Qn—m = (qo, 41,---;Qn—m—1) and consider 
the linear system Umu = Qn—mv + 0Qn—m.- 
Moreover, observe that max{qa(a): 2 € L(Qh_m)NS"~"} = An—m)- 


Now let us look at some consequences which will be needed in the sequel. 
First, recall: 
Let A € R"*” be a symmetric matrix, B = UL,AU, € R™*™ a section of A, 
and let 
ro > At >... > An—1 be the eigenvalues of A, 
No > AL >... > AMn_1 the eigenvalues of B, 
then: [Ajn—1;A0] C [An-1, Ao]. It seems that one can do better: 


Observation In the situation above, we have, more precisely: 
Xo = Xo 2 An—m, 
Ar > AM > An—m+1, 


Am—1 > Aim—1 > An-1- 


Proof Our minimax results immediately give what we want: For 1 < s < m we have 
actually: 

maxy, min{qa(x): 2 € L(Vs) MN S"7'} = Ag-1. 

maxy; min{qe(x): 2 € L(Vz)N S"~*} = Ag-1. 

Hence: 4,4 < As—1. 

On the other hand 

miny, max{qa(x) : 2 € L(Vs) MN S"7'} = Anes. 

miny, max{qe(x): 2 € L(V) S"“"} = Xn-s- 

Hence: X),,_; > An—s: 

This provides our claim. 


We are particularly interested in the following estimation: 


Consequence tr(U),AUm) < Ao AL +... + Am-1 
and this bound is attained for Um = Qm = (qo, q1,---Qm-1). 
When choosing U,, = (€0, €1,-.-@m—1) € R”*™, we obtain: 
doo ta11 +... + Am-1ym-1 < Ao t+ AL H+... +Am-1 1S m<n. 


Exercises 


(1) A symmetric matrix A € R"*” is positive definite <=> the quadric 


n-1 


qa(x) = ve Ar = S- Qj LiL; 


i,j=0 


only takes, for non-zero x, positive values (the eigenvalues of A are all positive). 
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Accept the following result: 


Coo 0 0 
; Cio Cl 0 
A = A’ is positive definite <=> there exists L = 
Cn—1,0 Cn—-1,1 Cn—-1,n-1 


subtriangular, with A = L- L* (extraction of a “matrix root”). 

(a) The matrix L (if it exists) can be computed column by column from the 
identity A= L- Lt 
(Cholesky algorithm). We obtain at the same time a criterion for the deci- 
sion whether A is positive definite. 
Find the formulas for the coefficients of L in case when A € R°*?. 

(b) Let g(2,y,z) = 2? +y +22 +aytaztyz. 


x 
Find the symmetric matrix A € R®*? such that q(a,y,z)=(ayz)Al y 
Zz 
Show, by the method of Cholesky, that A (hence q) is positive definite. 
Finally, diagonalize A: Find the orthogonal matrix Q € R?*? with 
Ao 0 0 
Q*AQ=D=1{ 0 A O 
0 0 A2 
Let A be a positive definite symmetric matrix, A = L- L* its Cholesky decom- 


position. 

True or false: The eigenvalues of L are the roots of the eigenvalues of A. 

True or false: A symmetric matrix with (at least) one negative element on the 
diagonal admits (at least) one negative eigenvalue. 

Let A be a positive definite symmetric matrix. Show that there exists a positive 
definite symmetric matrix C such that A= C?. 


5.2.2 The 2D DCT 


The real transformations in traditional digital signal theory are almost exclusively 
given by orthogonal matrices acting on (sample) vectors. The orthogonality is 
commonly considered as essential in order to guarantee the “preservation of en- 
ergy” — in more mathematical terms: one concentrates on isometries (preserving the 
Euclidean norms). It is above all in digital image processing that we are confronted 
with the problem of finding isometries acting on 2D schemes, i.e. on the matrices 
of the sample values of a digital image component. The mathematical construction 
which resolves easily this problem is the theme of this section. 


Tensor Products of Linear Transformations 


Tensor products of vectors. 


xo Yo 


Definition c= : a : ER”, 


In-1 Yn-1 
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XOYo TOY. «+» LOYn-1 
X1Yo TY. «+s L1Yn-1 
c@y=2ry = . . € R"*”, 
Ln—-1YO Tn-1Y1 --- Un—-1Yn-1 


Remarks (1) Let e; be the ith standard unit vector, 0<i<n-1. 
Then e; ® ej = Ei; = the matrix which is uniformly zero except at the position 
(i, 7), where the entry is equal to 1. 

(2) Every matrix X = (x;;) € R"*” can be written in a unique way as follows 


X= + Ligei ® e;. 


O<i,j<n-1 


(3) (Exercise) Let X € R”*” be a non-zero matrix; then there exists z,y € R” 
such that X =a2@®y <= > rg X= 1 (all the rows (columns) of X are 
proportional). 


Definition of a Euclidean structure on R”*”: 


The inner product: 
CG Yse Se eter 
O<i,jgn-1 


The Euclidean norm: 


IXl=VOGX =f So 23, 
O0<i,j<n-1 
) 


Observation (X,Y) =tr(XY* 
(Recall: The trace of a matrix is the sum of the elements on the main diagonal). 


Consequence (x @y,x' @y’') = (x,2')(y,y’) for 2,2’,y,y’ € R”. 
In particular: —|/x ® y|| = [||| - llyll - 


Proof {z@y,2' @y') = tr(zy*(2'y")*) = tr(zy’y'a2") = tr(wz"){y, y’) = (a, 2’) yy’). 


Exercises 


(1) Let {qo,qi} be the orthonormal basis of the R? given by 


1 (1 ey ee 
oe pd Be 2A) 


(a) Compute the four matrices qo ® qo, qo ® Q1, Qi ® Qo, qi ® Qh. 
(b) Show that these four matrices are an orthonormal basis of the R?*?. 
(2) Let {qo,qi,.--Qn—1} be an orthonormal basis of the R”. 
Consider the n? matrices qo ® qo, qo @Q1, ---; Qn—-1 ® Qn-1- 
Show that these n? matrices are an orthonormal basis of the R"™”. 
(3) Let Q € R"*” be an orthogonal matrix. Show that (QXQ*,Y) = (X,Q*YQ) 
for all X,Y € R"*”. 
Deduce that the transformation X+—~QXQ'‘ _ is an isometry of the R”™”. 
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Tensor Products of Linear Transformations 


Consider two matrices A, B € R”*”. 
Write A=(co,¢1,---,€n-1) and B=(co,c},...,¢,-1) 
(notation in column vectors). 
Let us define Tagp: R”"*" —>R”*”" by 


Taos(X) = S- Lig (Aex ® Be;) = S- LijCi ® c}. 
O<it,jgn-1 O<i,jgn-1 


Note that this is well-defined, since a linear transformation is uniquely determined 
by (linear extension on) the images of a basis; here: 
Tags (Ei) = ci @ cj O<ig<n-1. 


But: 
/ / / 
CoiCo; CoiCig +++ CO’ 1,5 
/ / / 
3 C1iCo; C1iCy 5 Pah C1liCn—-1,j 
Cc; ® Cc; = 
: / ; / . U 
Cn—-1,iCoj Cn—-1,iC1z +++ Cn—-1,iCn—-1,5 
This gives finally, for Y = Tagp(X), 
/ 
YRl = oshiza 4 Critig Cz, O<kl<n-1. 
Observation | Togq(X) = QXQ' (Q need not be orthogonal). 


Proof Write = Taaq(X) = (yet)o<ki<n—1 i.e. Yet = Sere ChiLij Cli, 
0<k,l<n-1. 

But pein Ceitiy = (QX)xj, hence eat CeiBigty = 
(QXQ")x1 as claimed. 


Remark Clearly, the observation above eliminates the tensor products that we have 
introduced so carefully. But certain (conceptual) properties of the 2D versions, in- 
herited from the characteristic properties of the 1D transformations from which 
they are derived, are best understood in tensor formalism. This will be a recurrent 
implicit argument in the sequel. 


Exercises 


(1) Let A, B,C,D € R”*” and let I, be the unit matrix of order n. 
(a) Show that Tacagp = Tags o Tcep and that Ty,,@1, = Idpnxn. 
(b) Deduce that, for invertible A, B, the transformation Tage is invertible, and 
we have: 


-1 
Prep = Ty-1@p-1- 


‘sh 
2 —3 
(2) TQ@Q @ 1 ) = = 


2 2 
Find the orthogonal matrices Q € R?**, which are thus determined. 


Nir 
Nl 


296 


(3) 
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For A = (aij), B = (bg) € R"*” let us define their tensor product (their 
Kronecker product, in old-fashioned language) A ® B € Rr xr? by 


aooB aniB ... daon-1B 
aioB ayiB Pug Q1n-1B 
A@®B= . . 
@n—1,0B an-11B ... Gn—1n-1B 


(we have to deal with a matrix of n? exemplaries of the second factor B, weighted 
by the coefficients of the first factor A). 


1 -2 1 2 
(a) a= (4, ) a= (57). 
Compute A@ B and B@ AE R*™*. 


(b) Write, for A = OOO | Mand = boo or € R?*? the matrix 
Q10 G11 bio bit 


A®BeER**%, with its 16 coefficients. 


(c) What is the relation between the transformation Tage : R"*" —> R"*” 
and the matrix 4@ BER” *” ? 
(Help: The question is finally the following: How to write a matrix X € R"*” 


as a column vector X° € R”’ in order to dispose of the following equivalence: 
Y=Taga(X) — Y°=(A@B)X*?). 

Let H(n) € R?"*?" be the matrix of the (non-normalized) Walsh-Hadamard 
transform to the order 2”. Show that H(n) = H(1) ® H(1) ®-:-® H(1) (n 
times). 


The 2D DCT 


The DCT acting on 8 X 8 matrices 


Recall The DCT of order8 Cg:R® —> R® 


Xo Yo 
K— ¥ 
x7 U7 
is given by 
7 
5 j(2k + 1)r 
w= a Sor cos i6 0<j<7 
k=0 
Ww forj = 0 
[with eS 
1 else 


7 
27 +1)k 
2 = >" - yp cos O24 Vin O<j<7 
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Consequence of the previous section: 


The 2D DCT Teac 


Loo... 


LETO: aie 


is given by the formulas 


Tm Tn (27 + 1)ma 
pe eet 


: R&x8 R&8x8 


XO7 


X77 


j(2m + 1)r 


k(2n + 1)r 


7 7 
mn = DO Dy Bik C08 16 


j=0O k=0 

Exercises 
5 Raa 
11 
11 
al 

(1) Compute Tcac 11 
11 
1 bea | 
11 
00 
01 
02 
03 

(2) Compute Toac 04 
05 
06 
07 


Pe RPP PRR 
el oe 
WO RR RRR RR Re 
Pe RP ERP Re 
el oe 


oO FR RRP RP PRE 


0 0 0 
4 6 7 
6 8 101214 
6 9 1215 18 21 
8 12 16 20 24 28 
10 15 20 25 30 35 
12 18 24 30 36 42 
14 21 28 35 42 49 


a 


< Sy fs 
16 ; 0O<m,n<7 


|X|] 0...0 
0 0...0 


(3) Determine the X € R°*® such that Tog@c(X) = 


0 0...0 


(“the energy of the transformed matrix is concentrated in the leading position” ). 


The DCT in JPEG 


The most popular standard in image compression, JPEG (“Joint Photographic Ex- 
perts Group”), is designed to work in several modes. 


298 5 Data Reduction: Lossy Compression 


The mode in which we are interested here is the mode of lossy compression. And 
it is exactly the 2D DCT which paves the way for the suppression of (secondary) 
information. 

More precisely: The operational scheme of the standard JPEG — in lossy mode 
— consists of four steps: 


(1) Sampling: Digital representation of the image (mostly photographic); the data 
units will be 8 x 8 matrices “covering” in three digital layers” image “tiles” of 
8 x 8 pixels. 

(2) (Local) intervention of the 2D DCT to the order 8 x 8: decorrelation of the 
numerical data, data unit by data unit. 

(3) Quantization: Annihilation of the “light” coefficients in the transformed matri- 
ces. 

(4) Compaction of the quantified data via Huffman coding (or via arithmetic cod- 


ing). 


Why is the Discrete Cosine Transform particularly appropriate to the decorrelation 
of the considered numerical data? 

The following section will give a precise answer to this question. 

But the pragmatic practitioner can also reason like this: 


11111111 80000000 
00000000 
00000000 
00000000 
00000000 
00000000 
00000000 
00000000 


We know that Tcac 


PR eee ee 
Bee ee ee 
a 
PRP ee Pee 
Pe RP eee 
Pee ee ee 
ee oo 
BPR eee ee 


aE 


Hence, a constant data unit is transformed into one leading coefficient which is 
its “energy” (the DC coefficient) and into 63 zeros (the AC coefficients). 

More generally, consider an arbitrary data unit: It consists of an 8 x8 matrix of 64 
sample values, where every single value encodes the intensity of a colour component 
(or of a derived coordinate) associated with a pixel. 

The scheme of the sample values: 


X00 LO1 --- LO7 

10 411 .-- L17 

X70 L71 .-. L77 

After 2D DCT of order 8 x 8, we obtain the transformed scheme 

Yoo Yol --- Yor 

Y1o Y11l - ++ Y17 

Y70 Y71 --- Y77 

We know: So 23; =o yn (the DCT preserves the Euclidean norm). 


2 According to the chosen colour system (we suppose a transformation RGB —- 
YCbCr). 
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Now write 


Loo Loi .-- Lo7 mm...m 600 O01 --- O07 
10 V1i1... X17 mm...m 610 O11... O17 
“270 L711... X77 mm...m 670 071... O77 


where m = +400 is the arithmetical mean of the x;;, and the 6;; are precisely the 
deviations from this average. This yields, after transformation, the decomposition 


Yoo Yol --- Yo7 80... 0 0 Yoi --- Yo7 
Y10 Y1l +++ Y17 0 0...0 Y10 Y1l ++. Y17 
yd : a ee . | ae : 
Y70 YU71 --- U77 0 0...0 Y70 Y71 --- U77 


As a consequence, the DC coefficient of the transformed scheme only depends 
on the average of the sample values, whereas the AC coefficients only depend on the 
deviations of the sample values from this average. 

This roughly explains the names direct current/alternative current for the coef- 
ficients. 

Note that (since the DCT preserves the Euclidean norm) a small mean square 
value of the AC (transform) coefficients equals a small mean square value of the 
sample variations. 


43 44 46 51 57 65 72 76 512.3 —15.2 66 2.9 —1.8 —0.5 —0.3 0 
44 45 46 50 56 63 69 72 —79.3 —51.8 7. 0.6 06 0 —1 —0.6 
47 47 46 49 55 60 64 65 45 —10.5 0.8 —2.5 0.7 04 06 0.3 
Example 53 53 52 52 54 57 59 59 a —6.6 5.3 —-1.1 0.2 01 O1 0.2 —0.6 
62 62 59 58 58 58 57 55 0.8 —0.7 0.3 —1.1 —-0.3 0.3 0.1 0 
75 73 71 68 67 66 62 60 0 0.4 0.1 —0.7 0.1 0.2 —0.3 0 
85 85 83 80 77 74 72 70 0.3 —0.8 01 15 0.3 —0.1 0.8 0.2 
92 91 91 87 85 83 81 80 —12 01 -0.3 0.3 0.1 —0.2 —1.1 —0.1 


Considering the example above as generic, we can state: If the values 7; are 
“numerically connected”, then we can expect that the values y,; are numerically 
important in the leading positions of the left upper corner of the matrix transform 
scheme, i.e. that we have to do with a value distribution which has approximately 
the form 


ABBCCDDD 
BBCCDDDD 
BCCDDDDD 
CCDDDDDD 
CDDDDDDD 
DDDDDDDD 
DDDDDDDD 
DDDDDDDD 


where A, B, C, D indicate, in decreasing order, the numerical weight of the 
position. 

Note that this situation seems to be a pictorial commonplace: a data unit de- 
scribes a local region of 8 x 8 pixels in the given digital image. Now, in photographic 


300 5 Data Reduction: Lossy Compression 


images, local uniformity of colour (or of the same Grey intensity) should be, in 
general, frequent: there will be sky, walls, doors, red cheeks, lawn ... 

All this will be transformed into data units which are “numerically homoge- 
neous”. 


Exercise 


Let A be the 8 x 8 matrix all coefficients of which are equal to 128, and let X be 
a 8x 8 matrix such that ||A — X|| = 16 (we admit an average deviation of size 
2 per coefficient). Let Y be the 2D DCT image of X. Show: We have for the DC 
coefficient: 1008 < yoo < 1040 and for the AC coefficients: yw < 256 — te. in 
average | yizj |< 2. 


The compaction algorithms (Huffman coding or arithmetic coding), which work 
in the final step of the JPEG compression, need, in order to be efficient, a great 
statistical imbalance of the data (of the letters to be treated). In other words: We 
need an important invasion of zeros in the DCT data units. 

This leads to a first (somewhat naive) idea of clever information suppression: 

If we annihilate the 48 coefficients of the transformed scheme which are outside 
the upper left 4 x 4 square, we will lower — for an image zone which is not too 
“agitated” — the Euclidean norm (the energy) of the entire scheme perhaps around 
5%. After retransformation of the “truncated scheme” we shall obtain a slightly 
changed initial square: The new numerical values of the 64 coefficients will differ 
from the initial values in average 5% each. If the colour palette is tolerant, this will 
not change drastically the quality of the image. 

The annihilation of the “light” coefficients will actually be done more carefully: 
One uses quantization tables: 

qoo qo1 --- 407 

dio Qi1 +++ G17 


q70 Q71 +++ Q77 
The “planing down” is then done when passing to the quantized coefficients: 


where [] means the round to the nearest integer. Two examples of quantization 
tables: 
A. Luminance: 


16111016 24 40 51 61 
12121419 26 58 60 55 
141316 24 40 57 69 56 
1417 22 29 51 87 80 62 
18 22 37 56 68 109 103 77 
24 35 55 64 81 104113 92 
49 64 78 87 103 121 120 101 
72 92 95 98 112 100 103 99 
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(Note that the luminance Y varies a priori? between 0 and 255, then will be 
renormalized between 16 — the Black threshold — and 235). 


B. Chrominance: 


17 18 24 47 99 99 99 99 
18 21 26 66 99 99 99 99 
24 26 56 99 99 99 99 99 
47 66 99 99 99 99 99 99 
99 99 99 99 99 99 99 99 
99 99 99 99 99 99 99 99 
99 99 99 99 99 99 99 99 
99 99 99 99 99 99 99 99 


(the chrominance — Cb and Cr — varies a priori between —127.5 and 127.5, but 
will be renormalized between 16 and 240, with 128 as “zero value”). 
These tables have been established empirically. 


Example In the following example, we consider the treatment of a data unit of lumi- 
nance sample values. Note that the luminance component will be the only component 
in the treatment of a Black and White digital photo (the chrominance components 
concern the colours). We shall display the compression as well as the decompression 
(the encoding and the decoding in the compaction step will remain in a black box 
— we refer the reader to the first chapter of this book). 


The Grey sample values The DCT coefficients 

139 144 149 153 155 155 155 155 1259.6 —1.0 —12.1 —5.2 2.1 —1.7 —2.7 1.3 
144 151 153 156 159 156 156 156 22.6 —17.5 —6.2 —3.2 —2.9 -—0.1 0.4 -—1.2 
150 155 160 163 158 156 156 156 —10.9 —9.3 -16 1.5 0.2 —0.9 —0.6 —0.1 
159 161 162 160 160 159 159 159 -7.1 -19 02 15 09 -—0.1 0.0 0.3 
159 160 161 162 162 155 155 155 —0.6 —0.8 1.5 1.6 —0.1 -—0.7 06 1.3 
161 161 161 161 160 157 157 157 1.8 —-0.2 16 -—-0.3 —-0.8 1.5 1.0 —1.0 
162 162 161 163 162 157 157 157 1.3 0.4 0.3 -1.5 -0.5 1.7 1.1 —0.8 
162 162 161 161 163 158 158 158 —2.66 1.6 -3.8 -18 19 1.2 —0.6 —0.4 
The quantization table The quantized coefficients 

16111016 24 40 51 61 79 0 -100000 

12121419 26 58 60 55 —2-1 0 00000 

1413 16 24 40 57 69 56 —1-1 0 00000 

14 17 22 29 51 87 80 62 -1 0 000000 

18 22 37 56 68 109 103 77 0 0 0 00000 

24 35 55 64 81 104113 92 0 0 0 00000 

49 64 78 87 103 121 120 101 0 0 0 00000 

72 92 95 98 112 100 103 99 0 0 0 00000 


3 A priori: computed, together with blue and red chrominance, via linear transfor- 
mation, from the colour coordinates Red—Green—Blue, which vary between 0 and 
255. 
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The dequantized coefficients The reconstructed Grey values 
1264 0 -—1000000 142 144 147 150 152 153 154 154 
—24-12 0 00000 149 150 153 155 156 157 156 156 
—14-13 0 00000 157 158 159 161 161 160 159 158 
-14 0 0 00000 162 162 163 163 162 160 158 157 

0 O 0 00000 162 162 162 162 161 158 156 155 
0 O 0 00000 160 161 161 161 160 158 156 154 
0 O 0 00000 160 160 161 162 161 160 158 157 
0 O 0 00000 160 161 163 164 164 163 161 160 


Let us explain the operations that we have carried out: 


(1) The table of the Grey sample values X describes a relatively uniform region: 
The sample values don’t vary sensibly. 

(2) The DCT coefficients: We have computed Y = CgXC§. 

(3) The quantized coefficients: We have divided every coefficient of the matrix Y by 
the corresponding coefficient of the quantization table, then we have rounded 
to the next integer. 
~The table of the quantized AC coefficients is read sequentially, in zigzag: 

0, —2,-1, -1,—-1,0,0,-1,—-1,0,0,... and will be encoded by a compaction 
algorithm (Huffman or arithmetic coding). The DC coefficients are encoded 
separately, along the data units. This ends the compression. 

The decompression begins with the decoding of the compacted data: One recon- 
structs the table of the quantized coefficients. 

(4) The table of the reconstructed DCT coefficients: We have multiplied every quan- 
tized coefficient by the corresponding quantizer (1, 264 = 79-16, —24 = —2-12, 
etc.). 

(5) The matrix X’ of the reconstructed Grey values: Let Y’ be the matrix of the 
reconstructed DCT coefficients; then X’ = CfY’Cs. 


Let us finish this section on the intervention of the 2D DCT in JPEG witha 


Remark The DCT can justly be considered as a decorrelation transformation which 
does not change the character of the data that it treats: What is temporal, remains 
temporal, what is spatial, remains spatial. The next section, treating the DCT as 
a KLT, will reinforce this statement. Conceptually, there are then no frequencies 
indexing the positions in the DCT coefficient table. On the other hand, as we have 
already pointed out, the direct current/alternative current terminology seems to be 
perfectly adapted. Moreover, the DCT is intimately linked to the Discrete Fourier 
Transform which decorrelates time domain data into frequency domain data. Will 
there be altogether “high frequencies” in DCT tables? 

Let us briefly look at this question. 

As an exercise, we shall replace, in an implementation of JPEG, the matrix Cs 
of the DCT by the following matrix Qs: 
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i 1 1 1 1 1 1 1 
2V2 0 2Y2 BY 2YD YB 2D 2Y2_ 22 
2/14 2714 2714 2V14 2V14 2V14 2V14 2V14 
0 6 1 it 1 i 1 1 
42 42 oD) 42 42 V42 a2 
0 0 5 1 1 1 1 1 
Qs = 30 30 30 30 V30 V30 
8 0 0 0 4 i i i i 
2/5 2Vv3 2VY5 2/5 25 
0 0 0 0 2V3 2v3 2/3 2/3 
0 0 0 0 Dae % Ve 
0 0 0 0 0 0 =e os 
1...1 
This matrix is orthogonal and it diagonalizes the same matrix A= | : as 
ileoreeee | 


the DCT Cs. 
We get an equivalent decorrelation transform, but every “frequential flair” has 


now disappeared. 
What about the effect of either transformation on samples of high variation? 


0 250 0 250 0 250 0 250 
250 0 250 0 250 0 250 0 
0 250 0 250 0 250 O 250 
250 0 250 0 250 0 250 0 
0 250 0 250 0 250 0 250 
250 0 250 0 250 0 250 0 
0 250 0 250 0 250 O 250 
250 0 250 0 250 0 250 0 


We get, for the C’g — values and for the Qg — values, the two matrices: 


Example Consider the sample matrix 


1000 0 0 0 i) 0 0 0 1000 0 0 0 0 0 i) i) 
0 —32.5 0 —38.3 0 —57.4 0 —163.3 QO —142.9 123.7 —146.4 119.5 —154.3 109.1 —189 
0 i) 0 0 0 0 0 0 0 123.7 —107.1 126.8 —103.5 133.6 —94.5 163.7 
0 —38.3 0 —45.2 0 —67.6 0 —192.6 QO —146.4 126.8 —150 122.5 —158.1 111.8 —193.6 
0 0 0 0 0 0 0 0 0 119.5 —103.5 122.5 —100 129.1 —91.3 158.1 
0 —57.4 0 —67.6 0 —101.2 0 —288.3 QO —154.3 133.6 —158.1 129.1 —166.7 117.9 —204.1 
0 0 0 19) i) 0 0 1) 0 109.1 —94.5 111.8 —91.3 117.9 —83.3 144.3 
0 —163.3 0 —192.6 0 —288.3 0 —821.1 0 —189 163.7 —193.6 158.1 —204.1 144.3 —250 


We note: Whereas the Qs — transform matrix reflects faithfully the value oscil- 
lation of the sample matrix, the Cs — transform matrix displays this information 
differently: the “high frequency” AC coefficients in the last row (and column) are 
plainly dominant. 

So, we have to accept: Despite the geometrical background of our decorrela- 
tion transform — find principal axes for a highly degenerate quadratic form — the 
DCT version allows, by its Fourier descent, a kind of traditional frequency domain 
description. 

By the way: Looking at the performance of JPEG(Qs), compared with the per- 
formance of JPEG(Cs3), we shall realize a small advantage of JPEG(Cs), which stems 
from the fact that the quantization tables have been established for the DCT. 
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Exercises 


(1) Compute the transformed matrix of 


in JPEG(Qs) version. 
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139 144 149 153 155 155 155 155 
144 151 153 156 159 156 156 156 
150 155 160 163 158 156 156 156 
159 161 162 160 160 159 159 159 
159 160 161 162 162 155 155 155 
161 161 161 161 160 157 157 157 
162 162 161 163 162 157 157 157 
162 162 161 161 163 158 158 158 


Quantize according to the luminance quantization table of JPEG(Cs). 


The transformed scheme 


The quantized coefficients 


1259.6 8.9 2.7 —1.5—-6.2 —8 00 79 1 0 00000 
20.6 10 —8.6 —6 —4.2 —3 00 2 -1-100000 
13.9 —9.4 —5.2 —5 —3.7 —0.700 1-1 0 00000 
7.6 —7.5 —4.3 —0.1 2.7 —1.700 1 0 000000 
—0.6 —2.2 —0.8 0.3 —2.4 —3.100 0 0 000000 
3.3 —-12-05 1 15 2 00 0 0 0 00000 
2 —0.1—-0.1 0.7 O —1.400 0 0 0 00000 
05 02 02 03 19 0 00 0 0 000000 


(2) Compare the transforms of 


in JPEG(Cs) and in JPEG(Qs). 


The transformed scheme in JPEG(Cs) 


1080 0000000 
—546.60000000 

0 0000000 
—57.1 0000000 

0 0000000 

-17 0000000 

0 0000000 
—4.3 0000000 


30 30 30 30 30 30 30 30 
60 60 60 60 60 60 60 60 
90 90 90 90 90 90 90 90 
120 120 120 120 120 120 120 120 
150 150 150 150 150 150 150 150 
180 180 180 180 180 180 180 180 
210 210 210 210 210 210 210 210 
240 240 240 240 240 240 240 240 


The transformed scheme in JPEG(Qs) 


1080 0000000 
317.50000000 
275 0000000 
232.40000000 
189.70000000 
147 0000000 
103.90000000 
60 0000000 
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5.2.3 The Karhunen—Loéve Transform and the DCT 


This section furnishes a conceptual complement for the assessment of the DCT. The 
1D Discrete Cosine Transform will be presented as the most ordinary member‘ of 
the family of orthogonal transformations reassembled under the generic name KLT. 

The Karhunen—Loéve Transform (KLT) is the solution of the following problem: 
Find a universal method of Linear Algebra which, in the outset of probabilistic signal 
theory, works efficiently for the suppression of secondary information (redundant or 
not), and which is optimal in a natural mathematical sense, inspired by a Euclidian 
(i.e. mean square error) formalism. 

Our language will thus be that of random variables. It should be pointed out 
that in this grey zone of descriptive Applied Mathematics all concrete algorithmic 
glory depends on an efficient statistical evaluation of the data. 


The Karhunen—Loéve Transform of a Random Vector 


Now we will have to refer to our knowledge of the diagonalization of real symmetric 
matrices, in a language of probabilistic modelling. 


Diagonalization of the Covariance Matria 


The situation: 


Xo 
xX 
Let X= . be a vector of n random variables. 
Xn-1 
Its mean is defined by 
mo E(Xo) 
My E(X1) 
mx = — 7 ’ 
Mn-1 E(Xn-1) 


where E(X;) is the expectation of the random variable X; 0<j<n-1. 
Its covariance matriz: 


900 001 O0,n-1 
O10 «O11 O1,n-1 
Ixy = ; 
On-1,0 FOn-1,1 +++ On—-1,n-1 


where oi; = E((Xi — mi)(X; — m;)), O<t,7 <n-1. 
In order to avoid formal pedantries, we shall always tacitly assume that the mean 
mx and the covariance matrix I’y are well-defined. 


Remark Concerning the expectation: At this rather general stage, any reasonable 
definition of a mean is acceptable, provided it satisfies the following two conditions: 


(i) Linearity: E(AiX1 + A2X2) = Ai E(X1) ad A2E(X2). 


* This is not a common opinion in signal theory. 
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(ii) Monotony: X < Y = E(X) < E(Y). 


Lemma The matrix Ix is positive semidefinite: 
Uo 


u'Txu>0 forall w= : ER”. 


Uo 
U1 


Proof (uo, U1 feos Un-1 ) (E((Xi — mi)(Xj — m;)) 


Un—-1 
= EP wi(X; — m,))?) > 0. 
Consequence The eigenvalues of the matrix Ix (which is a real symmetric 
matrix) are all non-negative. 
Diagonalize Ix: 
There exists a orthogonal matrix Q € R”*” such that 


a 
Ix =Q 0 — Q, 
0 0. ae 
where 
(i) 08 > of >--- > o2_, > O are the eigenvalues of x 
(ii) the columns qo, qi,.--,; Qn—1 of Q are an orthonormal basis of the R”, composed 


of eigenvectors for I’x. 


Definition With the notations introduced above, put 


Zo Xo — ™Mo 

Zi Xi —my, 
Z= =qQ'.(X—mx) = 

Zn-1 Xn-1 — Mn-1 


Z is called the KLT of the random vector X. 


Xo 
Xi 
Formal observation Let X = 7 be a vector of n random variables. 


Xn-1 
AER”"*",bER” 
Then: 


(1) max. = Amx +b 
(2) Paap SAPS A! 


5.2 The Discrete Cosine Transform 307 


Consequence mz=0 and Iz= 


2 
On-1 


? is the variance of the random variable Zj;,0<j<n-1. 


In particular, oj 


Auxiliary result Let Y be a random variable such that of? = 0 (the variance 


of Y is zero). 
Then Y = my almost surely (this is a consequence of Tchebyshev’s inequality). 
So, we can affirm: 
If o2_, =--- = 02_, = 0 (the rank of I'x is equal to n—r) 
then Zn-r = --: = Zn-1 = O almost surely. 
The number of the “degrees of freedom” of a random vector X is thus given by 


the rank of its covariance matrix I"y. 


The Optimality of the KLT 


Let us first point out the distinctive particularity of the KLT: 

Every random vector X has its own KLT, which depends on (the diagonalization 
of) its covariance matrix I’x. We insist: In the transformation equation 
Z=Q‘(X—mx), the coefficients of the transformation matrix Q* depend on X. 

In what sense is the KLT optimal? 

First, recall: 

Let f(0) be a (continuous) periodic signal, and let on f(@) be its Fourier poly- 
nomial to the order N. 

Then on f(9) minimizes a certain Euclidian distance: 

on f (0) is the orthogonal projection of f(@) (in an appropriate space of functions) 
onto the subspace of the trigonometric polynomials of degree < N. 

Here, we are in an analogous situation. 


Consider the vector space Y” of n-component random vectors X = : : 
Xn-1 
with mx = 0 (modulo the subspace of those which are almost surely zero). 


Our general hypothesis: The mean mx of X and its covariance matrix [’x will 
always be well-defined. Note, moreover, that the KLT acts only on zero mean random 


vectors. 
Introduce now the following inner product: 
Xo Yo 
GY) =20°2, 4) for X= Y= er. 
Xn-1 Veet 


We obtain for the associated Euclidean norm: 
= 4) 
|X|)? = (XK, X) = EQS, X?) = tx, 
where the trace of the covariance matrix I’x is equal to the sum of its eigenvalues. 
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Xo 
Now fix X= : . Let A= Tx be the covariance matrix of X. 
Xn-1 
Choose m = the order of truncation! <m<n. 
We are interested in orthonormal representations to the order m 


Zo Xo 
z=| : |=uhx=ut] : | ofx, 
Lit. Xn-1 
where Um = (Uo, U1,---,Um—1) € R"*™ is a matrix with m orthonormal 


columns (of length n). 

Note that to every orthonormal representation X +— Z = Ut,X corresponds 
a section Ut, AUm € R™*™ of the covariance matrix A = Ix of X. 

Xo 
In order to compare the random vector X = 
Xn-1 

with its “orthonormal truncation to the order m” Z = U;},X, we shall use the 

natural isometric section Y™ —>+ Y” given by (the multiplication with) Um. 


[The identity Uf,Um=Imn implies that ||UmZ|| = ||Zl| for all Z € ce 
The result which states the optimality of the KLT is the following: 


Proposition The situation as described above; then we have: 

|X — UnZl| is minimum for Um = Qm = (qo,qQ1,---;Qm-1), where 
{qo,q1,°-: ;Qn-1} ts an orthonormal basis of eigenvectors, corresponding to the 
eigenvalues 02 > 07 > ++» > 02_, > 0 of A=Tx. 


Proof ||K — UmZ|? = (X,X) — 2(X,UmZ) + (UmZ,UmZ). 

But (X,UmZ) = (X,UnU},K) = (U},X,UR,X) and (UmZ,UmZ) = 
(U},X, Ut, X). 

Hence: ||X — Um Zl? = |[XI|? — ||US X?. 

Finally: |[K — UmZl|? = trl’ — tr(ULIxUm) = 30") 0? — 7"," 97, where 
oi >of >--- > o7_, > O are the eigenvalues of A = Ix and oe > oe St DS 
am > 0 are the eigenvalues of B=U},IxUm. 

Our previous results on the extremal properties of the eigenvalues permit us to 
conclude. 


Let us sum up: 


Xo 0 
Let X = : be a random vector with mx = : 
Xn-1 0 

Let Z = Q*X be its KLT. 


Then: 


(1) The components of Z are completely decorrelated: 
O17 = cov(Z;, Z;) =0 for ifAj 
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(2) Consider, for each fixed m, 1 < m <n, the transformations by orthonormal 
truncation to the order m X +> U},X (as explained above). Then the difference 
of the “energy norms” ||X||? — ||U%,X||? is minimized by the Karhunen—Loéve 
truncation X +> QtX. 


The DCT — a Karhunen—Loéve Transform 


The Discrete Cosine Transform is the simplest KLT. In order to understand its 
strong position in concrete applications, recall the meaning of the notion “random 
variable”: Algorithmic information on the concrete values of the signals is replaced 
by (hopefully sufficient) information on the behaviour of the signals, thus permitting 
satisfactory predictions. You have to accept an environment where precise informa- 
tion comes from descriptive estimation, together with statistical evaluation. 

Now, the KLT of a random vector X is parametrized by its current argument, 
i.e. it is of the form Z=KLTx(X). 

In other words, in order to be able to transform an actual candidate X, we 
need a certain amount of statistical information which allows the computation of its 
covariance matrix (the diagonalization of which demands a pretty supplementary 
numerical effort). 

So, the KLT is, alas, not predefinable (and thus without interest for massive 
practical real time applications). 

But there is a small family of random vectors which has a common predetermined 
KLT: 


Xo 
Recall Let X = : be a random vector such that Ixy = 
Xn-1 
11...1 
d Real epee | 
11...1 
n0...0 
00 0 
Then xy =Ct | . : Cn 
0. ...0 
We are manifestly in the situation of a KLT with Q* =Cn: 


Xo — ™Mo 


Let then Z=C,(X—mx)=Cr : 
Xn-1 —Mn-1 


n0...0 


00 0 
We have: Iz = 
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Zo 
0 
Hence we obtain: Z = . almost surely. 


0 
Remark It is immediate — by retransformation of the result above — that a random 


vector X the covariance matrix Ix of which is constant, is necessarily (simply) 
repetitive: 


Xo 

Let us sum up: The DCT is the KLT of an extremal case (the case of “highest 
correlation” ). It is predefinable. 

In practical applications, one can always use the DCT as a brute force approxi- 
mation of the KLTs which actually ought to intervene, hoping that it will honestly 
replace them, at least in constellations of high correlation. More precisely: One 
hopes that the value ||X||? — ||CinX||? will not be too far away from the minimum 
|X|? — ||Qt,XI||? given by the KLT of X. 

Note that there exist rigorous approximation results. 


Exercises 


Our theme will be the Discrete Sine Transform. 


(1) Some auxiliary results. 


Recall Four trigonometrical identities: 
lee ce, Cee 
n e : _ Sin sin 
i sin(jz) o> sin 5 
(nt))e 
2 


n . cos 2£.sin 
a cos(jx) = ==! gig 


n + 27 _n cos(n+1)z-sin(nz) 
at sin (jz) 2 2sin x 


‘gem cos? (jx) = = 4 cos(n+1)z-sin(nz) 


2sin x 


Making use of the formulas above, show that 


0 for kFl, 
ju sin 44 sin 2 = ‘ 1<kl<n. 
ae for k=l, 
(Do not forget: sin x - siny = $(cos(x — y) — cos(x + y))). 
Ca Se ARR 1G 
sin $ sin = 
(2) Consider the matrix S2 = Jz eae = a (; zy) ; 
sin = sin = 


(a) Show that S»2 is an orthogonal (symmetric) matrix. 


(b) Let T2(a) = fe ci € R2*2, 
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Compute the matrix D2 = S2T2(a)Se. 
(c) Show that Ay =1—2acos$=1—a_ and 
A2=1 ices ttt 
are the two eigenvalues of 7>(a), and find an orthonormal basis {qi,q2} of 
eigenvectors for T2(a). 


(3) Consider now the matrix S4= ee 


sin 22 sin & gin up sin te8 
An 87 127 167 


Put a = sin =, B=sin 2 
(a) Write S4 in function of a and of £. 
(b) Show that S4 is an orthogonal (symmetric) matrix. 


1 -a 0 O 

(e) eras | eRe 
0 -a 1 -a 
0 0 -a 1 


Compute D4 = S4T,(a)Sa € R*** and verify that 
Ai =1-— 2acos 2, 

A2g = 1— 2acos nee 

A3 = 1— 2acos one 

Aa = 1— 2acos = 

are the ee of Ti(a). 


Conclusion The four columns of Sa are an orthonormal basis of eigenvectors 
for every matriz Ts(a), ae R. 


Generalize: Let n > 2 and let 4) saan a (sin IE ny 1 <5, k<n 
(a) Show that S, is an ary (symmetric) matrix. 
1 -a0 0... 0 
—-a 1l-a0... 0 
0 -a 1 —a 0 
(b) Let Tr(a) = ee . : be the n x n 3-band matrix, as 
1 -a 
0 0 0 -a 1 
indicated. 
Air 0... 0 
O r2 0 
Show that S,Tn(a)Sn = : , 
0... O An 
where Ax = 1 — 2acos A 1l<k<n. 
Xo 
Consequence Let X = : be a random vector such that x = T,(a), 
Xn-1 


for acertaina ER. 
Then the Discrete Sine Transform is the associated KLT. 
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Attention In signal processing, the situation which calls for the Discrete Sine 

Transform, is initially not a genuine vector setting. In other words, it does not 

stem from parallel thinking, but rather from modeling of sequential structures: 
Xo 


X= : will be the initial segment of a process (which has been trun- 


Xn-1 
cated after n steps). 

In case of a stationary first-order Markov process, the matrix T,,(a) comes 
up in a non-causal set-up of the situation (and is not a covariance matrix). On 
the other hand, the matrix T,(a)~* will be, up to a scalar factor, the covari- 
ance matrix of the residual process Y obtained by a decomposition of X in a 
deterministic part (the prediction) and a non-deterministic part (the prediction 
error) which is precisely Y. 

The Discrete Sine Transform is then the KLT of Y (if S, diagonalizes T;, (a), 
then S,, diagonalizes also T;,(a)~*). 


Now let us look at some clarifying exercises: 


Algebraic prelude. 
3 
2 


pp 

1 p 4x4 . ce 
° p 1 ER with pA. 
pp p 


Show that A is then invertible, and that A~* = —, 


(it is a symmetric three-band matrix). 
First conceptual invasion. 


Recall A white noise is given by a sequence of random variables (or of random 
vectors) (€n)nez, with zero mean: E(en) = 0,n € Z, of same variance: o”, and 
mutually non-correlated: cov(€m,€n) =0 form#n. 

One often tacitly assumes them Gaussian: €n ~N(0,1) for alln€ Z  (N(0,1) 
= the Gaussian law of mean 0 and variance 1). 

Now consider a first-order (stationary) autoregressive process: 

Xn =p: Xn-1+€n, n>1 with |p|<l. 

Our hypotheses: 

(i) (€n)n>1 is a white noise. 

(ii) E(Xn) = 0 for all n > 0. 
(iii) cov(Xn, Xn+n) only depends on h € Z. 
(iv) cov(Xm,€n) = 0 form <n. 

The term p- Xn—-1 corresponds to a (determinist) prediction, and the term en 
describes the (non-determinist) prediction error. 

It seems that this is the simplest model for speech signals and 1D scan lines of 
images. 

Xo 


Let now X= : be a truncation to the order n of such a process. 
Xn-1 
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Show that 
1 p p? pve 
: p 1 po ie pe 
o 1 n— 
Ix = p -?p 
a p? : : : 
p” 1 p” 2 oe 1 


(where o? is the variance of the considered white noise). 


(7) The non-causal formalism. 
(a) Verify that the process (Xn)n>o of the preceding exercise can also be written 
in the following form: 


Xn =a-(Xn-1 + Xn41) + en; n= 1, 


where a= ie and en, = Trp2 (en a pént1) ‘ 
(b) Show that E(e,) =0 and that 6? = E(e?) = mp forn21. 
e1 
(c) Show that we have, for e=| : 
En 
1-72; 0 0 
p p 
Rap ~~ T+p 1 ~~ 1+p? 0 
0 0 eo 1 
(a symmetric three-band matrix). 
1 -a 0 0 
; : _|-a 1 —a 0 ea 
(8) Consider the matrix Ti(a) = Gee ages, Ie R 
0 0 -a l 


(wherea= >fz and | p|< 1). 

For which values of p is the matrix T,(a) invertible? Compute Ty '(a). 
(9) Still the situation of Exercise 7. 

(a) Verify the identity 


X1 el aXo 
Xo a e€2 
Ti(a) X3 _ e€3 * 0 
X4 e€4 aXs 
aXo 
(b) Put X,=T,'(a)b with b = ; 
aXs 
Xp», the boundary response, is the determinist prediction which resolves the 
equations 
Xn = a(Xn-1 + Xn+1) according to the boundary conditions given by 
aXo 
0 
P= 0 
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Find the four components of X, in function of Xo, X5, and a. 


(c) Now consider Y = X — X, = Ty ‘(a) 


€4 
Show that X, and Y are orthogonal and that Iy = BT; *(a) ; 


5.3 Filter Banks and Discrete Wavelet Transform 


The practical interest of the transformations which will be presented in this last sec- 
tion comes from their appearance in JPEG 2000 — which is the successor of JPEG 
(just like AES is the successor of DES). So, we are still in search of decorrelation 
methods for our numerical data: the role of the DCT will now be occupied by the 
Discrete Wavelet Transform, which will efficiently create a kind of data hierarchy, ac- 
cording to a high resolution/low resolution display of the digital image. The objective 
will be, as before, to establish the appropriate numerical structure for quantization, 
i.e. for an intelligent suppression of “what seems to be unimportant”. 

Our presentation will be similar to that of the preceding section: First, we shall 
give a functional description (“how it works”), then we shall deal with design argu- 
ments (“why it works well”), i.e. with the underlying Mathematics. 


5.3.1 Two Channel Filter Banks 


Let us first point out that the Discrete Wavelet Transform (DWT) is actually a 
whole family of transforms — exactly as in the case of the KLT. 

In order to understand the algorithmic functioning of a Discrete Wavelet Trans- 
form in image compression, it is almost preferable, as a first approach, not to know 
what a wavelet is (more precisely: what a multi-resolution analysis is). For the novice, 
the (two channel) filter bank formalism seems to be the right access: One faces simple 
and robust (exclusively algebraic) mathematical structures, the algorithmic aspects 
of which are easily assimilated. As a first intuitive orientation: the filter banks are 
to Discrete Wavelet Transforms what formal power series are to convergent power 
series. 


Perfect Reconstruction Filter Banks 
Deconvolution 


Recall. In the first section of this chapter, we have encountered the mathematical 
formalism for the treatment of digital n-periodic signals (which, for n > 2m, are 
precisely the faithful samplings to the order n of trigonometric polynomials of degree 
<m). 

Our first result was the following: consider, for an invariant linear filter T’ on 
the discrete n-periodic signals, h = T(eo), the impulse response of the filter, i.e. 
|| : ho, Ai,..-,hn-1: || = Tl]: 1,0,...,0: ||). Then T(x) = x *h for all input signals 
x (the filtering is done via convolution with h = T(eo)). 
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The associativity of the convolution product means that the impulse response of 
a composition of two invariant linear filters is precisely the convolution product of 
their individual impulse responses; clearly, eo is the impulse response of the identity 
filter. 

In the domain of the n-periodic digital signal, the perfect reconstruction (or 
deconvolution, i.e. the invertibility of an invariant linear filter) has become a familiar 
topic for us: 


Consider h = || : ho, hi,...,hn—1 : ||, then there exists g = || : go, g1,---;9n—1 : || 
with gx h=h*xg=e0 every component of 
ho 
< hy 
h=r(Ay=| 
hed 


is non-zero. 
In this case, 


Gn-1 


Hence, in the theory of n-periodic digital signals, there are a lot of perfect 
reconstruction filters. Unfortunately, the periodic formalism is of minor practical 
interest. 

The non-periodic digital signal theory gets its natural Hilbert space formalization 
as follows: 

Consider then 1?(Z), the vector space of (complex-valued) sequences f = (f[k]) kez 


such that 
S- | fA] P< 00 
keZ 


We insist: f[k] = the value of f “at the moment k” (the discrete time “is” Z). 

That is our Hilbert space” of “finite energy” discrete signals. 

For the practitioner, this formal setting is clearly too general. So, consider the 
subspace of the digital signals with finite support: 


2 (Z) = {f = (f[k])nez: f[k] =0 for almost all k} 


A digital signal is an element of 12(Z) whenever it takes only a finite number of 
non-zero values. For formal convenience, let us introduce a variable t, which will be 
the “discret-time carrier”: an element f € [2(Z) thus becomes a Laurent polynomial 
f= > f [Ks 


Note that we also shall have negative powers of t!) 


° Recall the definitions of the Chap.3. At this stage, we would not substantially 
make use of the inner product and of the norm. 
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Example Consider f with 


f[k] =0 for k < —3, 
f[-2)=-1, f[-1] =2, 
f(0) = 1, f{1] = 9, 

f [2] = -3, f[3] = 1, and 
f[k] =0 fork > 4. 


Then f t-?7 42t7' 41-37 +2. 


Attention The variable t does not take values. Its unique mission is to position 
the values of the digital signals in time (or in space). ° 

The convolution product f « g of two digital signals with finite support f and g 
is defined by 


fxeln]= >> fleloli] = 5° flelgln — k] = S— fn — k]glh. 


In other words: the polynomial notation of the convolution product is the poly- 
nomial product (of the notations) of the factors. As in the case of the periodic digital 
signals, we consider the right-shift operator o: 


(of)[k] = f[k — 1). 
Clearly, this operator corresponds to the multiplication by t on the polynomial 
notations. 


Exercise 


For n € Z, let e, be the characteristic function of {n} C Z. 
In other words, 


Ls: se, 
en[k] = ‘a else. 
(a) Show that {e,,n € Z} is a basis of the vector space /2(Z). 
(b) Show that em * €n = Emin. 
(c) Show that of = e; «f. 


Remark The multiplication by t shifts the values of the signals one position to the 
right, while translating the discrete time one unit to the left (k + k— 1). If you 
want to repair this formal discordance, replace t by z~! . And you have got the 
z-transform of the considered digital signal! 

But let us return to our theme: the deconvolution — now in the non-periodic 
case. Let then T be an invariant linear filter on the finite support digital signals (T 
commutes with the operator a), and let h = T(eo) be the impulse response of T. 

Then T(f) = f*h = hxf (the filtering is a simple convolution with h). The proof 
is the same as in the n-periodic case (Exercise). We insist: in polynomial notation, 
a linear and time-invariant filtering is nothing but the multiplication by a fixed 


® We have already encountered this situation when formalizing convolutional codes. 
But, in binary arithmetic, there was no temptation to evaluate the variable. 
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polynomial h. The problem of deconvolution in the (finite support) non-periodic 
case now reads like this: 

Given a (Laurent) polynomial h, find a (Laurent) polynomial g with hg = 
gh=1. 


Exercise 


The only Laurent polynomials which are invertible for the multiplication (of Laurent 
polynomials) are the monomials of the form at”, n € Z, a #0. 


Consequence The only invariant linear filters on finite support digital signals 
which admit a perfect reconstruction (i.e. which are invertible as operators on finite 
support digital signals) are the shift operators (combined with scalar multiplica- 
tion).’ How can we repair this shortage of invertible filters? 


First Appearance of a DWT 


Arithmetical observation In Z, only the two integers 1 and —1 are invertible 
for the multiplication. In order to get a lot of invertibles for the multiplication in 
integer arithmetic, you must either pass to arithmetic modulo n or pass to integer 
matrix arithmetic. For example, the multiplicative group GL2(Z) of the 2 x 2 integer 
matrices of determinant +1 is sufficiently rich; look: 


Cy Ga) = 


Looking for analogies in discrete signal theory, we observe that the n-periodic 
case (corresponding to arithmetic modulo n) is indeed richer in perfect reconstruc- 
tion filters than the non-periodic case. But what about similarities when passing to 
matrices with polynomial coefficients — and how can we interpret a matrix impulse 
response? Let us begin with an example which will be important for the continuation. 


Exercise 


Consider the two matrices 


10 
01 
which are mutual inverses. So, we are in a situation of perfect reconstruction. But 
there remains the question: What is the action of a matrix impulse response on a 
digital signal (of finite support)? 


Verify that AS=SA= ( . We have got two “matrix impulse responses” , 


” Traditionally, deconvolution will produce an infinite length inverse, which has to 
undergo Procrustean truncation — this yields a kind of imperfect reconstruction. 
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A 2x2 matrix transforms couples into couples. Hence we have to create a filtering 
situation, where the input signal enters via two subbands and where the output signal 
leaves in two channels. Actually, for the analysis matrix A, the two input channels 
will be virtual, due to a certain interleaved lecture of the input signal, whereas the 
two output channels will be real — there will be two separate subbands. For the 
synthesis matrix S, the two input channels will be real, whereas the two output 
channels will be virtual: one reconstructs the initial signal in its interleaved lecture. 

More precisely: 

Let x=) > 2[nJt” be the input signal for A (in polynomial notation). Introduce 
an (even/odd) interleaved lecture which creates two virtual subbands: 


Xo = Ys x[2n]t” 
x1= LS x[2n + 1]t” 


(Pay attention to discrete-time (or discrete-space) positioning: we carry out a 
down-sampling of factor 2). 
One obtains 


where 


Yo = >) yo[nJt” = D7 y[2n]e” 
= Yo yi[n}t” = So y[2n + 1e” 
are the low-pass and high-pass components (take this as a definition) of the 
output signal 
y= by y[n|t” of A 


(here, it is the unifying interleaved notation which is a little bit artificial. ..). 
Look at our example: 


yo\ S= y[2nje” Z $— 2(¢+t7') (+t) So a[2nje” 
my) VS ybedne) Vtg 4 So a[2n + 1Je” )’ 


y1 = So ylen + 1]t” =  72RnIt" 7 Pnlt tt 1405 x[2n + 1]t”. 


Comparing the coefficients of t” in both cases, we obtain: 


y[2n] = —Za[2n — 2) + 


y[2n + 1] = —}2[2n] z[2n + 1] — ;2[2n + 2] 
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In non-interleaved version we have, for the low-pass component yo and for the 
high-pass component yi of the output signal: 
yo[n] = { — ga[k — 2] + $x[k — 1] + 32[k] 4 
yiln] ={-ae[k-1] +3 


We note: Finally, we have got rid of the matrix formalism. The matrix A is 
replaced by two ordinary filters with impulse responses® 


alk +1) — $a[k + I} on 


hj = (H5[-2], 5-1], ASLO], ASL] SBD) = (— 3. 9p peg) 
and 
i = (Mi[-1}, AIO), AVL) = (5, 5-9) 


The input signal x = (a[n]) splits into a low-pass component x * hj and a high- 
pass component x * hj, which give, by down-sampling of a factor 2 — in the sense 
mentioned above — the definite values of the two subbands yo and yi. 

The two components yo and yi admit a unified lecture, in interleaved version, 
as y = (yln]) with y[2n] = yo[n] and y[2n + 1] = ysln) . 


Exercise 


In the situation above, we know that there is perfect reconstruction: 


xo\ SS x[2n]t” 7 1 —$(1+t) So y[2n]t” 
xy NS eine Vee) ete) S> y[2n + i]t” J 


Find the two impulse responses sp and s,; such that 


Xo=y*So and x1=y*Si1. 


[50 = (s0[—1], s0(0], soll]) = (-5,1,-5), 
s1 = (s1[-2), s1[-1], #110], ill], s1f2)) = (- 3, 55+ 5-9) 


This is our first example of a two channel filter bank (or: a subband transform). 

Note that we have committed a “formal clumsiness” concerning our two recon- 
struction filters: as well so as s; take their input values in zigzag lecture from the two 
subbands of our filter bank; on the other hand, they reconstruct the initial signal 
x in two (virtual) separate subbands. Conceptually, the reconstruction should be 
done by two (impulse responses of) synthesis filters gj and gj, defined separately on 
the low-pass channel and on the high-pass channel, and such that the values of the 
initial signal are obtained by simple addition of the results of these two filterings. 

More concretely, we aim at a reconstruction of the form 


ak] = bs y[2i]go[k — 2é] + S- yl2i + IJgi[k — 21-1] 


We note: in interleaved reading, gj and gf will act alternately on the values of y. 


8 ()' for “translated” will distinguish the interleaved notation from the traditional 
notation. 
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Exercise 


Show that in the case of our example, the two synthesis filters gj and gi are 


ai = (ail-1], 95101, 9b 11) = (5.1, 5) 


and 


3) i, 2b 
2 


gi = (oi[-2], gil-1], gil0), gill] sf 2) = (-3,-5.5)- 5-9): 


Now let us display the usual scheme of a two channel filter bank: 


x(k] «ho 12 yo|n] T2 *gO 
| { 
*hi 1 2) yi[n] — [f 2 *91 x(k] 


Analysis filter bank Subbands Synthesis filter bank 
| 2: down-sampling of a factor 2 
T 2: up-sampling of a factor 2 


(In the first case, we only sample for even arguments (fx = f[2k]), 
in the second case, we put the sample values on even positions while filling with 
zeros on odd positions (fox = f[k], for4i = 0).) 


Attention The formalism according to the scheme above differs slightly from the 
formalism that we have adopted. The interleaved notation and the down-sampling 
(up-sampling) of factor 2 harmonize for ho and go. In the case of hi and gi, there 
will be, for counting arguments, a shift of one position. 


Let us explain, following our example; we get 


yiln] = { walk 1] + =2|k] Zolk+ Wheonte i.e. 


1 
2 
yal] = { - zolk] + 520k +1) - Folk + Doon 


This gives 
hi = (hi[—2], hi [—1], A [0]) = (—4, 4, —4) im traditional notation. 


Exercise 


Find, in the case of our example, the traditional notation for gi. 


3574) = (1-1, 91[0], 91 [1], 1 [2], 91[3)). 


Let us sum up: the traditional notation for h; begins one position lower, that 
for gi begins one position higher than the notation of the interleaved formalism. 

For algorithmic arguments, the interleaved formalism is much more natural — 
and we shall adopt it in the sequel. The traditional formalism is more natural when 
using standard transformations (the «transform, the Fourier transform), since it 
decomposes our subband transforms nicely into its primitive constituents. 


1d 
Answer g1 = (—4,-4; 


Nolo 
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Exercises 


We shall try to find the natural generalizations for our generic example. All impulse 
responses are to be read in interleaved formalism. 


(1) 


Let (hg[k]) and (hj [k]) be the analysis filters of a two channel filter bank: 
yl2n] = J ho[k|e[2n— k], yf2n +1) = So Aifk]e[2n + 1-4] 
k k 


Let 

> apt® > byt” 
a > crt > dt” 
be the corresponding analysis matrix. 
Show that then 


C= hi [2k =e 1] dy = hi [2k]. 


Convention Throughout the rest of this section, we shall always tacitly as- 
sume to be in real signal theory. 


The matrix A is invertible (we get perfect reconstruction) <= det A is invertible 
(as a Laurent polynomial) <=> det A = at” for a € R*,n € Z . Show that a 
given analysis filter bank (hj,h{) admits a synthesis filter bank (g},g}) for 
perfect reconstruction <=> >, ho[k] - (—1)*hj [2n — k] is non-zero for precisely 
one value of n. 

Note that we can always suppose that det A = a for a € R*. 

We will have then: a = )*, ho[k] - (—1)*Ai[-A]. 

(The argument is simple: Multiply each of the two rows of the matrix A by an 
appropriate power of t, and the determinant will become “constant” ). 

In the sequel, we always shall suppose to be in this situation. 

Assuming that det A = a € R*, the synthesis matrix has to be 


eu 2 So dyt*® — > b,t* 
a\—SSext® Sraxt® )° 
Conclude, for so, si with xo = y * So and x; = y * Si that 
80[2k] = dk, s0[2k — 1] = —+b,, 
si[2k +1] =—4cx, 51 [2k] = tag. 


In the situation of the preceding exercise, let us go a little bit further: 
Show that the synthesis impulse responses gj and gj of a perfect reconstruction 
filter bank are obtained from the analysis impulse responses hj and hj as follows: 


gbln] = —(-1)" Ai In, 
gi ln] = ~(—1)" Ab In, 


where a = )>, ho[k] - (—1)*hy[-]. 
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(5) Consider an analysis filter bank given by (hj, h}), where 


hy = (ho[—4], ho[—3], ho[—2], ho[—1], ho [0], ho [1], 2o[2], 2013], hol4]) 
r (= 1 5 17 19 17 5 1 >) 
32’ 64’ 64’ 64’ 32’ 64’ 64’ 64’ 32/’ 


=(< 1 19 9 19 1 =) 
64’ 32’ 64°16’ 64’ 32° 64/° 


Verify if the couple (hj, hj) satisfies the conditions for the analysis impulse 
responses of a perfect reconstruction filter bank. 
(6) Let 


ho = (ho[-3], ho[—2], ho[—1], ho [0], hol], ho [2], ho[3)) 


= ( 3 3 73 17 73 3 =) 
280’ 56’ 280’ 28’ 280’ 56’ 280/” 


i = (hi[-2], AiL-1, AiO], ALL} ALD = (-35,-9, Ge - 3p) 


20’ 4’5’ 4° 20 


Verify if the couple (hj, hj) satisfies the conditions for the analysis impulse 
responses of a perfect reconstruction filter bank. 

(7) Let Ry, be the real root, and let Rz and R3 be the two other (conjugate) roots 
of the equation R(x) = 8 — 22+ 1027 — 22° =0. 
Let us define: 


5 
hp [0] = Jog (Ps — 1)(14 + 24R2R3 — 16(R2 + Rs)), 
5 
hof1] = ho[—1] = i9g (1 1)(12 + 16R2R3 — 14(Re + Rs)), 
5 
ho [2] = hb [-2] = jog Rt 1)(8+ 4R2R3 — 8(R2 + Rs)), 
5 
ho [3] = ho[—3] = Tag (Fa 1)(4— 2(R2 + R3)), 
ie oa _ 10 
ho 4 ho 4 128 (Ri 1); 
then: 
3Ri —2 
hi [0] = 
a 8(Ri = 1)’ 
t t 8Ri-7 
hy 1 ha 1] 32(Ry = hid 
t t _ 2—Ri 
mite 16(Ri — 1)’ 
t t 1 
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(a) Verify that 


DY holn] = 1, DV Ailn] =0, 
L(-1)"holn] = 0, D(—1)"Ai[n] = 1. 
(b) Verify that (hg, h{) satisfy the conditions for the analysis impulse responses 
of a perfect reconstruction filter bank. 


(8) We return to the situation of the exercise (4), with a = 1 (this is a modest 
normalization). 
Let us define two systems of vectors (€n)nez and (€n)nez as follows: 
i[k —n] n=0mod 2, 
en(k] = i ] 


gi[k—n] n= 1mod 2, 


hi[n—k] n=1mod 2. 


(Pay attention to the time-inverted progressions !) 
Show that 


5, [k] = ree k] n=0Omod 2, 


1, m=n 


O° las (inner product of 1?(Z)). 


(a) (@m;€n) = 


(b) Every x € 12(Z) admits a unique representation x = )> (x,@n)en- 


Help:(a) For the inner products (em,én), with m and n of the same parity, you 
should make use of the reconstruction identity 


>, Polk] - (—1)* A} [2n — &] = fe for n = 0, 


0, else. 
For the inner products (em, €@n) with m and n of unequal parity, you should use 
the following auxiliary result: 
>, (-)* - al[k]a[n — k] = 0 for all sequences (a[k])xez and every odd n . 
(b) Here, you only deal with a different notation of the identity 


x[k] =~ yolllas[k — 2) +S yi ldjgife — (20+ 1)] 


The Discrete Wavelet Transform — version JPEG 2000 


In this section, we shall give a purely functional presentation of the Discrete Wavelet 
Transform as a kind of decorrelation kit for the digital data in JPEG 2000. Actually, 
the DWT will appear as (a certain type of) two channel filter bank. 

We shall give a rather rigid definition of the type of filter bank which really 
interests the practitioner in digital image processing — and shall call it, by misuse of 
language, a Discrete Wavelet Transform. The criteria which allow recovering such a 
filter bank as a true Discrete Wavelet Transform — i.e. as a collection of operations 
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making sense in a continuous world — will be discussed in the last section of this 
chapter. 

Note that the two filter banks imposed by the first version of JPEG 2000 — the 
DWT 5/3 spline and the DWT 9/7 CDF — are true Discrete Wavelet Transforms. 


The 1D Formalism 


We shall restart almost from scratch with the presentation of a two channel filter 
bank, while insisting upon the aspect “treat the data by a window of fixed length”, 
which is — a priori — hostile to convolutional thinking. 

So, consider the following situation: 

x|n] = x[0]x[1]x[2]...a[N — 1] a sequence of N digital values. We shall suppose 
N to be even, often N = 1,2,4,8,..., i.e. a power of 2. We shall associate with the 
sequence x[n] two sequences yo[n] and y:[n], the length of which will be half of that 
of x[n]: 

yo|n] = the “low-pass” subband sequence 


yi[n] = the “high-pass” subband sequence 


In order to obtain a correspondence of sequences of the same length, pass now 
to the interleaved sequence y[n], with 


y[2n] = yoln], 


yl2n +1] = ys[n). 


ae oe x[0] x[1] x[2] x[3] x[4] x[5] x[6] [7] 
yO] yf1] yl2) yf3] yl4] yl5) yl6) yl, 
Yyo[0] yi [0] yolt] yi [1] yol2] y1[2] yo[3] yi [3]. 


In the sequel, each of the particular values y[n] will depend, via some fixed linear 
combination, on a “numerical neighbourhood” of the corresponding value x[n], for 
example on z[n — 2]a[n — 1Ja[n]a[n + 1Ja[n + 2] . 

We shall get formal evaluation problems at the boundaries. 

The remedy will be a representation by symmetric extension 


9. 


[n] = 2[n], O0<n<N-1, 


[N ri nj =X[N — 1+ n]. 


ps x0] x[1] x[2] x[3] 
. X{—2] X[—1] X[0] X[1] X[2] X[3] X[4] X[5] ... 
. «[2] ax[l] x[0] x[1] 2/2] 2 


a 
Ey 
NX 
8 

= 


° It is more frequent that a digital image will be cut up along “numerical plateaux” 
than along “numerical fractures”: this yields the idea of a symmetric prolongation. 
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How shall we pass concretely from the sequence x[n] to the sequence y[n], and 
(reconstruction!) from the sequence y[n] to the sequence x[n]? The relations which 
define the analysis transformation and the synthesis transformation will be formu- 
lated in symmetric extensions (i.e. on infinite periodic sequences): 


vy 


Yn] = Yo iez hn moa alt] ¥[n — i] (analysis) 


x[n] = ee Ylélg; mod 2 [n = i] (synthesis) 


Attention The sequences x[n] vary — hence the sequences y|[n] vary with them (the 
symmetries of ¥[n] will be the same as those of X[{n]). On the other hand, the four 
finite sequences hé[n], hi[n], g6[n], gi[n] are constant (they define “sliding windows” 
of weighting coordinates) and characteristic for the considered filter bank. 


We insist: 
The analysis filters: 


h§[n] = the low-pass analysis impulse response, 
h{[n] = the high-pass analysis impulse response. 


The synthesis filters: 


g}|n] = the low-pass synthesis impulse response, 
gi(n] = the high-pass synthesis impulse response. 


Concerning the ()' = translated notation: 

In a conventional-filter-bank formalism (we mentioned it in the preceding section, 
without further specification) we have four impulse responses ho, hi, go, gi, which 
are related to the translated variants in the following way: 

ho[k] = ho[k], Gok] = golk], 

hi[k]=halk-1], — gilk] = glk +1). 

The translated formalism has been chosen in order to obtain a natural situation 
relative to the interleaved notation. In the design of (two channel) filter banks for 
image compression we shall insist upon 

Linear phase: 


Ao[n] = ho[—n], 
Ay[n] = hi[-n], 
go[n] = go[—n}, 
gi[n] = gi[—n}. 


Hence our (translated) impulse responses always will have odd length and will 
be symmetric with respect to n = 0. 
The fundamental relations giving the invertibility of the analysis transform: 


Yo, hile] - (-1)* Ai [2n — k] = 


aéR*, forn=0, 
{ (R) 


0, else. 
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We shall then necessarily have (cf. the exercises at the end of the last section): 


goln] = <(-1)"hi [nm], 


giln] = 5(-1)"holn], 


where a is given by the condition of perfect reconstruction (R) above, and is 
nothing but the determinant of the analysis matrix A considered in the preceding 
section. By the way, we have, in a more frequential language (Exercise, for the 
incredulous): 


h® = S~ h*[n] 
dcep,n dcp,n 
a= 3 (ho yy + hy ho®’*) with pra — ae 

For the filter bank design (in the restricted meaning of this section), you thus 
only have to specify the (impulse responses of the) analysis filters h§[n] and h(n], 
satisfying the relations of perfect reconstruction (R). All the remainder will then be 
determined. 


Note We shall try to obtain, in general: 

née =1 beet =A. (this is a normalization) 

hie =0 ho’? =0 (this is a conceptual necessity) 

We insist: the cancellation condition is necessary in order to guarantee that our 
purely arithmetical constructions have a chance to stem from the wavelet universe 
(that they can be derived from operations in a continuous world). 

Finally, we shall have, in most cases: a = z. 


Commentary 


The condition hé° = 1 means that the low-pass analysis filter computes local 
averages — it will be a kind of integrator. 


The condition h{° = 0 means that the high-pass analysis filter computes generalized 
slopes — il will be a kind of differentiator. 


The information in the low-pass subband more or less concerns numerical 
regularities. 


The information in the high-pass subband rather concerns numerical irregularities. 


Example The DWT 5/3 spline. 
he = (-4,4,3,4,-4): hi = (-4,3,—-4)- 
That’s our “generic” example of the preceding section. 
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We will search for the matrix representation of the analysis filter on sequences 
of length 8: 
y(0) «(0] 
How can we obtain : in function of : ? 


y[7] «(7 


The general formula splits into low-pass filtering and into high-pass filtering in 
the following way: 


yl2n] = 22" F?_ xli]hG[2n — 4, 


i=2n—2* 
y[2n + 1] = Sent? xfajht [an +1—- 
More explicitly: 
y[0] = ¥[—2]h§ [2] + X[-1]hG[1] + X[0]AG[0] + ¥[1]AG[—-1] + ¥[2]hh[-2I, 
y(1] = X[OJh4[1] + X[1Jht [0] + X[2]h5[—-1], 
y[2] = ¥[0]h5 [2] + ¥[1Jh5[1] + ¥[2]h5[0] + ¥[3]h5[-1] + ¥[4]AG[—-2], 
y[3] = X[2]h4 [1] + X[3]h4 [0] + ¥[4]Jht[-1], 
y[4] = ¥[2]h6 [2] + ¥[3]h6[1] + ¥[4]h5 [0] + ¥[5]h6[—1] + ¥[6]AG[—2], 
y[d] = X[4Jh4 [1] + X54 [0] + X[6]hi[-1], 
y[6] = X[4]h§ [2] + X[5]h5 [1] + X[6]hG [0] + X[7]hG[-1] + ¥[8]hG[—-2], 
y[7] = X[6)h4 [1] + X[7]h4 [0] + X[8]hi[-1], 
But: X[-2]=2[2], X[-1]=<2[l], and  x{8] = 2[6]. 


This gives (the impulse responses are symmetric with respect to n = 0): 


y[0 hé[0] 2h§[1] 2h6[2] 0 O O 0 0 x[0 
y[l Ai[l] hifo] Ai) oO Oo O 0 0 x1 
yl2 h§ [2] h§[1] AG[0] AS] AS[2] 0 0 0 x[2 
y[3] | 0 0 AY] AL[O] hf [1] oO 0 0 z[3 
yi] ]~ | 0 0 ABI} ASE] ASLO] AS] = AQ] aA 
y[5 0 0 0 0 Ai AiO] = AL] 0 x5 
y[6 0 0 0 0 hg [2] [1] AG[0] + AG [2] hG[1] x6 
yl? 0 0 0 0 oO 0 2hi[1] Af [0] x(7 


Now let us substitute the concrete values; we obtain: 


y|0 2 +-10000 0 «(0 
yf -ti_1i9 0000 capt 
9 22) Be Ci ype G 2 
y 84 4,9 8 x 
y[3] |_| 0 0-42-30 0 0 a[3 
yf4] |} | 0 O-g 3 3 7780 a4 
y[5 000 0-45-70 z[5 
yl6 000 0-¢% 34 a6 
yl7 00000 0-44 a7 
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Consider, as a first example, a constant input sequence: z[n] = 1. We obtain: 


y|0 1 
y[1 0 
yl2 1 yo [0] 1 yi [0] 0 
y[3 _ | 0 . yo[1] } — f 1 yi[1] } _ { 0 
yi =|, i.e. yol2) | ~ | 1 and w2) | = | 0 
y[5 0 yo[3] 1 yi [3] 0 
y[6 1 
y|7 0 


Refine a little bit: consider an input sequence in linear progression: 
z[n] =n+1,0<n<7. 


This gives 

y[0 il 

yl 0 

y[2 3 yo(0] 1 yi [0] 0 
y[3] | _ | 90 yo[l] |_| 3 w(l] |_| 0 
yal |=} 5 1** Ly ft 5 | a? wl} =] o 
y[5 0 yo([3] 7.25 yi [3] 0.5 
y|6 7.25 

yl? 0.5 


We note: the low-pass channel displays the regularity of the sequence (a linear 
progression). The high-pass channel displays the absence of (numerical) gaps in the 
considered sequence. There are “numerical deviations” due to the treatment of the 
boundaries by symmetric extension. 


Now let us pass to the synthesis transform (inversion of the analysis transform). 


We have to make explicit the reconstruction formula 


x(n] = ee ylilgt mod a(n i_ i). 


goln] = 2-(—1)"Ay 
Recall een = 2-(-1)"hb In] 


In our case: 


9 = (5, 1, 3), 


t 1 1, 33. ab uF 
gi = ( 4.39297 2 a): 

When evaluating the reconstruction formula position by position, we obtain: 
x(0] ni[o]  —2hp [1] 0 ) 0 0 0 ) y(0] 
a([1] hi [1] AG [0] + PG[2] —h4i [1] h§[2] 0 0 0 0 y[1] 
«(2 0 —Ao[1] hy [0] —hg[1] 0 0 0 0 y[2] 
[3] | _, 0 Agi2] = —hi [1] AGlO] —ALf] rE] 0 0 y[3] 
24] 0 ) 0 —hb[1] AsO] —hG [1] 0 0 y[4] 
[5] 0 0 0 hol2] —Ai [1] holo] —hi[1] AG [2] yl5] 
x(6] 0 0 0 0 0 = —hAG[t] hi [0] —AG [1] y(6] 
27] 0 0 ) 0 0 = 2h6[2] —2h*[1] h§[0] y[7] 
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This gives concretely: 


x[0 1-1000000 y[0 
afl 4237-10000 y[1 
x2 o-i1-i0000 y(2 
23 pt Ee tt hg 3 

_ 42 3 32-4 y 
z[4]} | 0 0 0-51-50 0 y(4 
«(5 000-15 2 3-4 y[5 
z|6 0000 0-51-5 y[6 
a|7 0000 0-41 2 y[7 


For the incredulous: verify that the product of the synthesis matrix with the 


analysis matrix is indeed the unit matrix. 


More modestly: we clearly have: 


1 11-1000 000 1 
2 4 33-1000 0 0 
3 0-41-4000 0 3 

ee ae 

4] _ | 0-73 3 4-700 0 
5| |0 0 0-31-50 0 5 
6] |0 00-23 $3-4]| 0 
7 0000 0-41-53 725 
8 0000 0-41 2 0.5 

Exercises 


(1) 
(2) 


Let (a[k])-w<k<n be an arbitrary finite sequence. 
Show then that °,(—1)*a[kja{n—k]=0 for all odd n. 
hi = (-:, i, 3. i —3) (symmetric, of length 5). 
Write the equations (R) — with a = 4 —as a linear system for the computation 
of hy = (hi [—1], h} [0], hi [1]) (symmetric, of length 3) such that (hg, h}) will be 
the analysis filters of a perfect reconstruction filter bank, and recover by this 
method the DWT 5/3 spline. 
Let hj = (hb[—4],..., 2§[4]) be a (low-pass) analysis impulse response, symmet- 
ric, of length 9 (preferably with hg° = 1 and hj’? = 0). Consider the following 
linear system: 
holO]X — 2hg[UY + 2hG[2]Z — 2hG[3]U = a(= §), 
nS [2]X — (h[1] + ALB + (hb [0] + Ab (AI) Z — HSILIU = 0, 
nS[AJX — ALBIY + hS[2]Z — RS TJU = 0, 
h§[4]Z — h6[3]U = 0. 
Show that there exists hi = (h{[-3],...,h4[3]), a (high-pass) analysis impulse 
response, symmetric, of length 7 such that the ensuing analysis filter bank 
admits perfect reconstruction <> the linear system above admits a solution 
(X,Y, Z,U) = (hi [0], AX[1], hi [2], W4[3)). 
Consider the one-parameter family of symmetric ( high-pass) impulse responses 
of the form 
with 
hy [0] =a 
hi [£1] = -F 
hi [+2] = + - 
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We search for symmetric (low-pass) impulse responses!? of the form 


hy = (ho[—3], ho[—2], ho[- 1], Ao[0], Ao [1], Ao[2], ho[3]), 
such that (hj, hj) are the analysis filters of a perfect reconstruction filter bank. 


(a) Show that the low-pass partner is obtained when solving the following linear 


system: 
2a 1 1—2a 0 ho|0] 1 
1—2a 1 4a 1 ho[1] } _ | 0 
0 0 1—2a 1 ho[2] } | 0 
Le ee 2P 2 32 ho [3] 0 


(b) The system above admits a unique solution for every parameter value a 4 +. 
Give the solution in parametrized form. 


_ _14+4a __ —8a?+4+18a—5 

Answerit ho[0] = 4(4a—1) ho[1] = 8(4a—1) 
* a— 1—2a)(3—4a 
ho[2] = CS ho[3] = - Sar - 


(c) For a = 2 write explicitly (hp, h{) and (g, g1). 


The 2D Formalism 


Initial situation We dispose of a DWT (a two channel filter bank). 


Objective Define the associated (D level) 2D DWT. 

Let [nn] = z[n1, n2], 0<n,neo<N-1 

be a matrix!” of N? sample values (i.e. a component of a digital image). 

Our 2D DWT will transform it into an interleaved matrix of the following four- 
subband structure: 


bi, be E {0, 1} 


y[2n1 + b1, 2n2 + bz] = yo, 0,[N1, n2] with OSnicne < 1N _] 


Recall Our N should preferably be a power of 2: 
2, 4, 8, 16, 32,... 


How shall we get the matrix y(n] from the matrix «x[n]? 

First, we apply the given DWT WN times separately on the N columns of «[n]. 
Call the resulting matrix y’[n]. Then, in order to obtain y[n], apply our DWT N 
times separately on the N rows of y'[n]. 


' Do not forget the condition }>(—1)*ho[k] =0 . 

" Recall that hi = ho. 

!2 When dealing with matrices of sample values, we shall often skip the matrix 
brackets. 
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Observation Support from matrix algebra. 


=T-x 


Consider the matrix representation of our DWT: y 


T-X-T* |. 


Then we shall simply have, in 2D: |} Y 


We insist: the matrix Y = y[n] will be a triple matrix product, with the matrix 


x[n] as its central factor. 


X= 


(Generic) scheme: 


5) 


0 
1 
2 
3 
4 
5 
6 


a RO ln a 
SHNM TH IDOY 
“Sa SS "> 
ouwnovnovno no Oo 
SHAN wi tid oN 
“aD BD DB BD 


x 
x 
x 


6 
6 


? 


0 
1 


’ 


x 
x 


Wd 1D 1D 1D 1D 1D 1D 1D 
SHANOT DON 
“SDD DD DB BD > 
SS SSS SS 
SHNMDT DON 
“aa "> 
oD OD OD OD OD OD OD OD 
SHAN wi tid oN 
“a DD HS 
NANNANANAN 
SHANOM DON 
DDD DD DB BD > 
a Ore eet oat etl ort 
SHNMT DON 
“>"> DD DB DBD > 


SoSoSeosao]5 
SHANA tS ON 


x|0, 0] x|0, 1] x[0, 2] x[0, 3] x[0, 4] x[0,5 
x{1,0} x[1, 1) x{1, 2] [1,3] x[1, 4] x[1,5 


xz|2,0| x[2, 1) x[2, 2] [2,3] x[2, 4] x[2,5] r[2,6 


x|3, 0] [8,1] x[3, 2] x[8, 3] x[3, 4] x[8, 5] x[3, 6] x 
x4, 0] [4,1] x4, 2] x4, 3] x4, 4] x4, 5] x4, 6] x 
[5,0] x[5, 1] x[5, 2] [5,3] x[5, 4] x[5, 5] x5, 6] x 
x|6, 0] x|6, 1] x[6, 2] x|6, 3] x6, 4] x[6, 5] x6, 6] x 


x7, 0] x[7, 1] x[7, 2] [7,3] x[7, 4] x[7, 5] x[7, 6] x7, 


becomes, after transformation of the columns, 


SS Se SSS Ss 


HQ mA 


and finally 


y[0, 0] y[O, 1] yO, 2] y[0, 3] y[0, 4] y[0, 5} y[O0, 6] y[0, 7 
y(1, 0] y[1, 1] y[1, 2] y[1, 3] y[1, 4] y[1, 5} y[1, 6] y[1, 7 
y[2, 0] y[2, 1] y[2, 2] y[2, 3] y[2, 4] y[2, 5] y[2, 6] y[2, 7 
y[3, O} y[3, 1] y[8, 2] y/8, 3] y[8, 4] y[3, 5] y/[3, 6] y[3, 7 
y[4, 0] y/4, 1] y[4, 2] y/4, 3] y[4, 4] y[4, 5] y[4, 6] y[4, 7 
yl5, O} y[5, 1] y[5, 2] y[5, 3] y[5, 4] y[5, 5] y[5, 6] y[5, 7 
y|6, O} y[6, 1] y[6, 2] y[6, 3] y[6, 4] y[6, 5] y[6, 6] y|6, 7 
yl7, 0} y[7, 1) y[7, 2] y[7, 3] y[7, 4] y[7, 5] y[7, 6] y[7, 7 


L 


high-pass 


Now separate the four interleaved matrices: 


L 
val 
LL 


low-pass sub-band 


y[0, 0] y[0, 2] y[0, 4] y[0, 6) 


yoo[0, 0] yoo[9, 1] yoo[9, 2] yoo[0, 3] 


y[2, 0] y[2, 2] y[2, 4] y[2, 6] 
y[4, 0] y[4, 2] y[4, 4] y[4, 6] 


y[6, 0] y[6, 2] y[6, 4] y[6, 6]. 


3 
3 
3 


? 


1 
2 
3 


] yoo[1, 1] yoo[1, 2] yoo[1, 3] 
yoo[3, 0] yoo[3, 1] yoo[3, 2] yoo[3, 3] 


0 
0 


1 


yoo(1, 


] yoo[2, 1] yoo[2, 2] yoo| 


? 


2 


yoo| 
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HL= horizontally high-pass sub-band 


yor[0,0] yor[0, 1] yo1[0,2] yor[0,3] — y[0, 1] y[0, 3] y[0, 5] y[0, 7] 
Yol 1,0 You 1,1 Yo. 1,2 Yol 1,3 -_ y[2, 1 y|2, 3 y[2, 5 y[2, 7] 
yo1{2, 0} yo1[2, 1] you[2, 2] yor[2,3 y[4, 1] y[4, 3] [4,5] y[4, 7] 
yor[3, 0] yor[3, 1] yo1[8, 2] yoi[3,3] — y[6, 1] y[6, 3) y[6, 5] y[6, 7] 
LH = vertically high-pass sub-band 
yi0[0, 0] y10[0, 1] yro[0, 2] yro[0,3] —-y[1, 0] y[1, 2] y[1, 4] y[1, 6] 
yro[1, 0] yro[1, 1] yso[1, 2] yo[1,3] _ y[3,0] y[3, 2] y[3, 4] y[3, 6) 
yio[2, 0] yro[2, 1] yro[2, 2] yro[2,3] [5,0] y[5, 2] y[5, 4] y[5, 6] 
y1o[3, 0] yro[3, 1] yro[3, 2] yro[3,3] [7,0] y[7,2] y[7, 4] y[7, 6 
HH = high-pass sub-band 
yi1(0, 0} y11[0, 1] yia[0, 2] y11 [0,3 y[1, 1] y[1, 3] y[1, 5] yf, 7] 
yis[1, 0] yra[L, 1] yri[1, 2] yra[t,3] _ y[8, 1] y[3, 3] y[8, 5] yf, 7] 
yi1[2, 0] yaa [2, 1] yra (2, 2] yra[2,3] [5,1] y[5, 3] y[5, 5) y[5, 7] 
yi1[3, 0] y11[3, 1] yii[8, 2] y11[3, 3 y[7,1] y[7, 3] y[7, 5] y[7, 7]. 
Attention The notation for the horizontally high-pass and vertically high-pass 
sub-bands is (perhaps) surprising; you might have expected a “transposed” notation. 


Example Back to our DWT 5/3 spline 
(the name refers to the length of the impulse responses and to a design criterion). 
First: 


Lite 10101010 
beet Pai 00000000 
Piri 10101010 
is ik sas Pe Pee 00000000 
£141 4I 1 101041016 
LAT ad 00000000 
ie es ae 10101010 
Leta ee 00000000 


A constant matrix is “condensed” into LL. 
HL, LH and HH are zero. 


Then: 
11111111 101 010 1 0 
22222222 000 00 0 0 0 
33333333 3 0 3 0 3 0 3 0 
44444444 000 00 0 0 0 
55555555 #2450505 050 
66666666 000 00 0 0 0 
T7777777 7.25 0 7.25 0 7.25 0 7.25 0 
88888888 0.5 0 0.5 0 0.5 0 0.5 0 
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It is instructive to have a look on the four interleaved matrices. 


i ot. ae At 

ee ee ee es 

oo 5 5 5 5 

7.25 7.25 7.25 7.25 
000 0 
00 0 0 
d= 000 0 
0.5 0.5 0.5 0.5 


HH 


0000 
0000 
0000 
0000 


0000 
0000 


0000 
0000 


LL is an approximate lower resolution of the initial gradation. 
The vertically high-pass sub-band LH is (timidly) non-zero; this stems from the 
fact that our gradation is vertically non-constant. It is amusing that this absolutely 
satisfactory information comes from a mathematical ugliness: the oblique treatment 
of a linear progression because of its symmetric extension!!? 


Exercises 


(1) Recall: the 2D DWT 5/3 spline (to the order 8) transforms 


and 


X2 


i 

II 
eo 
Pee eee ee 
Pree eee ee 
oo 
oo 
Pe RP eee ee 

e 


11111111 
22222222 
33333333 
44444444 
55555555 
66666666 
TTTT7777 
88888888 


11 
11 
11 


11 
11 
11 
11 


into Yo= 


into Yj= 


Ot 
Oo: Oo: OO OO: O'S 


10101010 
00000000 
10101010 
00000000 
10101010 
00000000 
10101010 
00000000 


wPonowor 


ot 
oo OO oO .Oo Oo 
Or 
ooooooco 


o 
on 


ooooooco 
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13 Pay attention to pictorial interpretations of the “compressed image”: it is the ver- 
tically high-pass sub-band LH which will show a small horizontal bar on uniform 
Grey — your conclusion? 
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Deduce the transform of 


—0.375 —0.25 —0.125 0 0.125 0.25 0.375 0.5 
—0.375 —0.25 —0.125 0 0.125 0.25 0.375 0.5 
—0.375 —0.25 —0.125 0 0.125 0.25 0.375 0.5 
_ —0.375 —0.25 —0.125 0 0.125 0.25 0.375 0.5 
~ —0.375 —0.25 —0.125 0 0.125 0.25 0.375 0.5 
—0.375 —0.25 —0.125 0 0.125 0.25 0.375 0.5 
—0.375 —0.25 —0.125 0 0.125 0.25 0.375 0.5 
—0.375 —0.25 —0.125 0 0.125 0.25 0.375 0.5. 


X3 


(2) Consider an analysis filter bank (hg, h}) such that }> hi [k] = 0. 
Then show that the transform of a vertical gradation (all rows are constant) 
necessarily satisfies HL = 0 and HH = 0 (the horizontally high-pass sub-band 
HL and the high-pass sub-band HH are uniformly zero). 
Give, in case of the DWT 5/3 spline to the order 8, an example of an 8 x 8 
scheme which is not a gradation (i.e. which is a matrix of rank > 2) and such 
that its transform satisfies HL = 0, LH = 0 and HH = 0. 

(3) Compute, with the 2D DWT 5/3 spline to the order 8, the second and the third 
transform iterate of 


11111111 
222522 22:2 
33333333 
44444444 
55555555 
66666666 
TTTT7T7T77 
88888888 


101 01 0 1 0 0 0 0 0 0 0 0 0 

0 00 0 0 0 0 0 —0.5 05 —-0.5 05 —-05 05 —-05 0.5 
3.0 3 0 3 0 3 0 0.75 —0.75 0.75 —0.75 0.75 —0.75 0.75 —0.75 
0 00 0 0 0 0 0 -1 1 -1 1 -1 1 -1 1 
50 Se BO RR Ss ed os aS he “es. 8 
0 00 0 0 0 0 0 —1.53 1.53 —1.53 1.53 —1.53 1.53 —1.53 1.53 
7.25 0 7.25 0 7.25 0 7.25 0 2.02 —2.02 2.02 —2.02 2.02 —2.02 2.02 —2.02 
0.5 0 0.5 0 0.5 0 0.5 0 —1.69 1.69 —1.69 1.69 —1.69 1.69 —1.69 1.69 


0.44 0 0.44 0 0.44 0 0.44 

0.44 0 0.44 0 0.44 0 0.44 
0.03 0 —0.03 0 —0.03 0 —0.03 
PO: 2.70 eke <OF al 


0 
0 
0 
0 
~~ 9 
0 
0 
0 


0.05 0 0.05 0 0.05 0 0.05 
1.58 0 1.58 0 1.58 0 1.58 
0.3 0 —0.3 0 —0.3 0 —0.3 
1.85 0 1.85 0 1.85 0 1.85 
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(3) The general philosophy is that the low-pass sub-band LL contains a lower (but 
rather faithful) resolution of the digital image (we have carefully computed local 
means). 

Give an example (of a non-constant 8 x 8 matrix) such that the low-pass sub- 
band LL is constant (by the 2D DWT 5/3 spline to the order 8): all the digital 
information will be “in the details”. 


200 0 200 0 200 0 200 0 100 0 100 0 100 0 100 0 
0 200 0 200 0 200 0 200 0 100 0 100 0 100 0 100 
200 0 200 0 200 0 200 0 100 0 100 0 100 0 100 0 
0 200 0 200 0 200 0 200 _s 0 100 0 100 0 100 0 100 
200 0 200 0 200 0 200 0 100 0 100 0 100 0 100 0 
0 200 0 200 0 200 0 200 0 100 0 100 0 100 0 100 
200 0 200 0 200 0 200 0 100 0 100 0 100 0 100 0 
0 200 0 200 0 200 0 200 0 100 0 100 0 100 O 100 


4) Consider the DWT 7/5 Burt’* given b 
g ry 


ho = (ho[—3], ho[—2], ho[—1], Ao[0], ho [1], Ao [2], ho[3]) 


=( 3 3. 73 17 73 3 =) 
280’ 56’ 280’ 28’ 280’ 56’ 280/’ 
1 
20’ 


Bi = (hi [-2], Ai [1], hi], Wi [1], hi 2D) = ( 


gi = (gbl—21, 951-11, 950], 9511], 9520) = ( 


£1 = (9i1[-3], 91[-2], 91 [-H, 91 [0], 91 [1], 91 [2], 91 [3]) 


=f 3 3 73 17 73 3 55) 
140’ 28’ 140’14’ 280’ 287 140/° 


11iiii1i111 
22222222 
33333333 
Write the analysis and synthesis matrices to the order 8 44444444 
and compute the transformed scheme of the gradation 55555555 
66666666 
TTT7T7777 
88888888 


'4 We deal with the most distinguished member of a family of filter banks. 
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The two matrices: 


17 73 3 3 
28 140 28 +140 0 0 0 0 
Te 01. 1 1 
4 20 4 20 0 0 0 0 
3 1 17 73 3 3 0 0 
560 4 28 280 56-280 
1 1 3 1 1 
O rap a eS e Oe LD 
0 3 3. 38 17 73 3 3 ? 
280 56 280 28 280 56 280 
1 it 3 1 1 
Oi: MO OP gg ae: See Ye ae ag 
3 3 1 31 73 
0 0 0 280 56 4 56 =. 280 
0 0 0 0 0 -4+ -E 3 
10 2 5 
6 73 1 3 
Bo -E Fo 0 0 0 0 
1 31 1 3 
5. oR 5 38 0 0 0 0 
1 1 6 73 1 3 
10 2 #6 140 10 140 0 0 
BA 17 1 3 
0 28 «2 14 2 28 0 0 
0 3 1 73 866 73 1 3 
140 10 140 5 140 +10 = 140 
3 1 17 1 3 
0 0 28 «2 14 2 28 
3 1 i 0b 73 
0 0 140 10 2 10 140 
0 0 0 0 0 3 1 ante 


1 
1.24 0 1.24 0 1.24 0 1.24 0 
0.1 0 —0.1 0 —0.1 0 —0.1 0 
2.98 0 2.98 0 2.98 0 2.98 0 
000 0 0 0 0 0 
5 5 5 5 5 5 5 5 
000 0 0 0 0 0 
7.15 0 7.15 0 7.15 0 7.15 0 
0.7 0 0.7 0 0.7 0 0.7 0 
(5) We aim for a (rough) comparison between the performances of the DWT 5/3 
spline and of the DWT 7/5 Burt to the order 8! with respect to ensuing com- 
pression (after appropriate quantization). 

(a) If the coefficients of the 8 x 8 scheme Yo vary between —4 and 4, what will 
be the bounds of variation for the transform coefficients of Yi, in case of 
the DWT 5/3 spline and of the DWT 7/5 Burt? 

(b) Consider the following four test matrices: 


The transformed matrix: 


1 Concerning the order 8: we suppose tacitly a transformation of square sample 
schemes. This is by no means necessary — and often not even wanted: a rectangular 
scheme Yo of m rows and n columns will be multiplied (as a matrix) by Tm from 
the left and by T*, from the right — where T,, and T, are the matrices to the order 
m and n of the considered analysis filter bank. 


Ao = 


2 
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v6 


5 


1 3 1 3 
4 16 8 16° °2~~«216 8 16 33333333 
3 15 9 21 3 21 9 15 SRS RRR ESR 
16 64 32 64 #8 64 #32 64 siadaaas 
1 5 3 Cas Wet 3 5 g444444 
8 32 16 32 4 #32 16 32 835525228 
L§-&-£-i-h ao ete atts 
16 64 32 64 8 64 32 64 By = § 4 8 8 48 
Or 0 -.G>. Oe a SOS ceoae es 
1 5 3 Ge st. a9 3 5 gp4888848 
16 64 32 64 8 64 32 £64 a ie a a Oe 
pe ue Se a oe §33433 3 38 
8 32 16 32 4 32 16 32 88888888 
3 15 9 21 3 21 9 15 7 
16 64 32 64 8 64 32 64 00055000 
4.°3 1 1 _3 id 
“a4 § UBTET Scho a0 
Se Ce ee ee eee eee 
PPasea ba Gf eS tiiiiiat 
8-4 8 2 8 4 8 22232222 
ee eae er en ae ae 00042000 
1 i ey: ee ees ii 
: 0 sa 78-3 8, 4 00055000 
To de gy eth 28-1 3 00054000 
a ee oer er ete eet 2% 
8 4 8 8 4° 8 2 
First, find their transforms Ai, Bi, Ci and Dy 
by the DWT 5/3 spline as well as by the DWT 7/5 Burt. 
We obtain (the coefficients have been duly rounded): 
0.25 0 —0.38 0 —0.5 0 —0.36 0.03 
0 0 0 0 0 0 0 0 
0.13 0 —0.19 0 —0.25 0 —0.18 0.02 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 
0.14 0 0.21 0 0.28 0 0.2 —0.02 
0.03 0 0.05 0 0.06 0 0.04 0 
—0.25 0.01 —0.35 —0.01 —0.46 —0.01 —0.34 0.04 
—0.01 0 -—0.01 0 -—0.01 0 -—0.01 0 
—0.13 0 -0.19 0 -—0.25 0 —0.19 0.02 
Aes 0 0 0 0 0 0 0 0 
Oe» AO: aD -' O) 0 0 0 0 0 
0 0 0 0 0 0 0 (0) 
0.14 O 0.2 0 0.26 0 0.2 —0.02 
0.05 O 0.07 0 0.08 0 0.06 —0.01 
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0.36 —0.02 037 O 0.38 0.01 0.36 0.02 
—0.02 —0.02 —0.01 0 0 0.01 —0.02 0.02 
0.37 —0.01 0.12 O 0.1 —0.02 0.26 0.07 
a 0 0 0 —0.01 —0.03 0.01 0 0 
0.38 8©600 0.1 —0.03 —0.03 0 0.28 0.06 
0.01 0.01 —0.02 0.01 0 —0.02—-0.01 0 
0.36 —0.02 0.26 0 0.28 —0.01 0.3 0.05 
0.02 0.02 007 O 006 QO 0.05 —0.03 


0.35 —0.02 0.34 0 034 O 0.34 0.03 
—0.02 —0.02 0 0 0.01 0.01 —0.02 0.02 
0.34 O 0.15 —0.01 0.11 —0.03 0.26 0.09 
0 0 —0.01 —0.02 —0.04 0.01 0 0 
0.34 0.01 0.11 —0.04 0.01 —0.01 0.26 0.09 
0 0.01 —0.03 0.01 —0.01 —0.02 —0.01 0.01 
0.34 —0.02 0.26 0 0.26 —0.01 0.3 0.06 
0.03 0.02 0.09 0 0.09 0.01 0.06 —0.05 


B= 


—0.44 0.06 0.5 —0.06 —0.03 0 —0.28 —0.06 
—0.19 —0.19 0.13 —0.06 —0.03 0 0 0 
—0.34 —0.09 —0.36 0.13 0.42 —0.03 —0.05 —0.06 


Oe 0 0 -—0.09 —0.19 0.13 —0.06 —0.03 0 
0 0 —0.3 —0.09 —0.36 0.13 0.38 —0.13 
0 0 0 0 —0.09 —0.19 0.09 —0.13 
0.28 O 003 O —0.27 —0.09 —0.27 0.25 
0.06 OO 006 O 0.06 O —0.06 —0.25 
—0.33 0.01 032 0 £0.03 O —0.24 —0.09 
—0.18 —0.22 0.12 —0.03 —0.01 0 —0.01 0 
—0.35 —0.11 —0.25 0.12 0.29 0 —0.02 —0.09 
C= 0.01 0.03 —0.13 —0.25 0.12 —0.03 0 —0.01 
—0.03 0.02 —0.29 —0.13 —0.26 0.12 0.27 —0.09 
0 0 0.01 0.03 —0.13 —0.25 0.11 —0.09 
0.24 0.01 0.02 0.01 —0.27 —0.12 —0.22 0.27 
0.09 O 0.09 O 0.1 0.06 —0.12 —0.31 
0 O 0.06 0.13 0.5 —0.13 —0.06 0 
0 0 O 0 oO O 0 0 
0.06 0 0.12 0.11 0.5 —0.11 0.01 0 
He 0.13 0 0.11 —0.03 0 0.03 0.14 0 


0.5 0 0.5 0 05 O 0.5 0 
—0.13 0 —0.11 0.03 0 —0.03 —0.14 0 
—0.060 0.01 0.14 0.5 —0.14 —0.13 0 

0 O O 0 oO O 0 O 
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—0.02 —0.03 0.1 0.18 0.43 —0.15 —0.04 0 
—0.03 0 —0.02 0.01 0 —0.01 —0.03 0 
0.1 —0.02 0.19 0.14 0.45 —0.12 0.08 0 
0.18 0.01 0.14 —0.06 0.02 0.05 0.10 O 
0.43 #O 045 0.02 0.49 —0.02 0.43 0 
—0.15 —0.01 —0.12 0.05 —0.02 —0.05 —0.16 0 
—0.04 —0.03 0.08 0.19 0.43 —0.16 —0.07 0 

0 0 0 0 0 0 0 O 


2 
Il 


Note the “numerical neighbourliness” of the matrices transformed by the 


two DWTs. 
(c) In order to compare the compressed bit-rates in the two cases, we shall 
proceed as follows. 
Our quantization will associate with every (rounded) coefficient an 8-bit byte, 
the first bit of which encodes the sign of the considered coefficient. 
The compaction by arithmetic coding will be carried out on the bitstream of the 
7x 64 = 448 bits given by the (truncated) binary notations of the 64 coefficients 
in the matrix scheme to be treated. 
(Example: 0.34 < 0101011). 
First we need the value po = 72, 
“source bitstream” of 448 bits. 
Counting table (“statistically balanced” '*): 


where No is the number of the zeros in the 


Size of the coefficient a Nienbet ot saienatin one 
seven significant bits 

0.5<a 0+3 

0.25<a<0.5 1+ 2.5 

0.13 < a < 0.25 2+2 

0.06 <a < 0.12 341.5 

0.03 < a < 0.06 4+1 

0.02 <a < 0.03 5+ 0.5 

a= 0.01 6 

a=0 7 


We shall replace the compressed bit-rate by the inverse compression ratio, i.e. 
by (approximately) the entropy H = —poLog,po — (1 — po)Log,(1 — po). 
Compare — in our four test examples — the inverse compression ratio obtained 
when using alternatively the DWT 5/3 spline and the DWT 7/5 Burt. 

(6) Effect of local quantization on the decompressed image. 


We shall study the propagation — in synthesis — of a local error (at the position 
(i, j)). Choose four positions in the four sub-bands of the transformed image: 
LL: F4,4 HL: Fa.3 

LH: £34 HH: E33 

Compute, for the DWT 5/3 spline and for the DWT 7/5 Burt the four corre- 
sponding synthesized matrices. 

We note: the unit matrices E;,; are the matrix products of a column unit vector 
e; with a row unit vector ef. As a consequence, their synthesis transforms will 


16 We suppose the “residual parts” — after the first 1 in the binary notation — to be 
of equal probability. 
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be the matrix product of the ith column of the synthesis matrix with the transposed 
jth column of the synthesis matrix. 


Let Si,; (and Vie respectively) be the decompressed matrices in question. 


Then we obtain (with rounded coefficients): 


000 0 0 0 00 00 0 0) 0) 0 0. 0 
000 0 0 0 00 00 0 0) 0) 0 0 0 
000 0 0 0 00 00 0.01 —0.05 —0.12 —0.05 0.01 0 
S, , = 00 0 0.25 0.5 0.25 0 0 gf = 90 -0.05 0.25 0.6 0.25 —0.05 0 
44-000 05 1 0.5 00 44> 00-012 06 1.44 0.6 —0.12 0 
000 0.25 05 0.2500 00 —0.05 0.25 0.6 0.25 —0.05 0 
000 0 0 0 00 00 0.01 —0.05 —0.12 —0.05 0.01 0 
000 0 0 0 00 00 0 0 0 0 0 0 
000 0 0 0 00 00 0 0.02 0.05 0.02 0 0 
0 0 0 —0.13 —0.25 —0.13 0 0 00 0.01 —0.05 —0.13 —0.05 0.01 0 
000 —0.25 —0.5 —0.25 00 00 0.05 —0.26 —0.63 —0.26 0.05 0 
Sa, = 0090 0.75 1.5 0.75 00 gf = 90 -0.12 0.61 1.46 0.61 —0.12 0 
3.4 = 000 -0.25 —0.5 —0.25 00 3,4 = 00 0.05 —0.26 —0.63 —0.26 0.05 0 
0 0 0 —0.13 —0.25 —0.13 0 0 00 0.01 —0.05 —0.13 —0.05 0.01 0 
000 0 0 0 00 00 0 0.01 0.03 0.01 0 0 
000 0 0 0 00 00 0 0) 0 0 0 0 
0 0 () 0) 0 0 00 0 0 0 0 0 0 0 0 
0 0 0 0) 0 0 00 0 0 0 0 0 0 0 0 
0 0 0) 0) 0 0 00 0 0.01 0.05 —0.12 0.05 0.01 0 0 
8, 4 = 0 0-13 —0.25 0.75 —0.25 —0.13 0 0 g/ , = 0:02 —0.05 —0.26 0.61 —0.26 —0.05 0.01 0 
4,3 = 0 -0.25 -0.5 1.5 -—0.5 —0.2500 4,3 = 0.05 —0.13 —0.63 1.46 —0.63 —0.13 0.03 0 
0 —0.13 —0.25 0.75 —0.25 —0.13 0 0 0.02 —0.05 —0.26 0.61 —0.26 —0.05 0.01 0 
0 0 0 0) 0 0 00 0 0.01 0.05 —0.12 0.05 0.01 0 0 
0 0 0) 0 0 0 00 0 0 0 0) 0 0 0 0 
0 0 ) 0) 0) 0 00 ) 0 -0.02 0.05 —0.02 0 0 0 
0 0.06 0.13 —0.38 0.13 0.06 00 0) 0.01 0.06 —0.13 0.06 0.01 0 0 
0 0.13 0.25 —0.75 0.25 0.13 00 —0.02 0.06 0.27 —0.63 0.27 0.06 —0.01 0 
Sa q = 0 —0-38 0.75 2.25 —0.75 —0.3800 7s _ 0.05 —0.13 —0.63 1.47 —0.63 —0.13 0.03 0 
3.3 = 0 0.13 0.25 —0.75 0.25 0.13 00 3,3 = —0.02 0.06 0.27 —0.63 0.27 0.06 —0.01 0 
0 0.06 0.13 —0.38 0.13 0.06 00 0) 0.01 0.06 —0.13 0.06 0.01 en) 
0 0 0) 0 0) 0 00 0) 0 0.01 0.03 —0.01 0 0 0 
0 0 0 0 0) 0 00 0) 0 0 0 0) 0) 0 0 


Commentary Note that the transformed matrices only have an explicit pictorial 
meaning for the sub-band LL. The three other sub-bands LH, HL and HH bear in- 
formation on details. Now, a local error (by local quantization) concerning merely 
descriptive data will be decompressed into a pictorial patch. This is not very se- 
rious, as far as everywhere quantization creates some modest uniform numerical 


deviation. 


Resolution levels — interleaved perspective 


Let us continue with our example of the 2D DWT 5/3 spline, acting on the gradation 


OGBAIDTBRWNEH 
CANDOR W;NE 
OABANDWIKWNEH 
SCNIDWEWNH 
CBNoOWKRWNE 
CBANIDOTBRWNEH 
CANO RW;NE 
ABNWDWIKWNEH 


First some preliminary computations. 


5.3 Filter Banks and Discrete Wavelet Transform 341 


We are interested in matrix representations to the order 4, 2 (and 1) of the DWT 
5/3 spline: 


Applying the analysis formula to hé = (-$, i 3, i —3) and hi = (—4, s —4), 
we obtain, for the computation to the order 4, 
y(0) ho[0] 2h [1] 2ho[2| 0 «(0 
y(t} | _ f Ai) Ai [0] hj [1] 0 v(1] 
y[2] } | Aol2] holt] hol] + Aol2] ho [A] x(2] 
y[3) 0 0 2h} [1] hi (0) «/3] 


(you have only to copy the first four rows in the computation to the order 8, 
while observing that now x[4] = x[2]). 


Concretely: 
v0) ¢ $ -} 0) (210 
vl | fe ea 2) Pe 
y24) {-s 2 ¢ 2] | 22 
y[3] 0 0 -2 $ x3] 


In the same spirit, we easily obtain the matrix representation to the order 2: 


yl0] \ _ ( ho[0] + 2h6[2] 2ho[1] \ ( x[0] 
y[1] 2hi[1] ha lO} 7 \ ef] ) 


a) eg ( 5 i) ey) 
y[1] St eat] 


Finally, we will have, to the order 1: 


Concretely: 


y[0] = (2ho[2] + 2ho[1] + ho[0]) - [0] = x{0]. 


For the corresponding synthesis transformations (to the order 4 and 2) we obtain: 


x(0] hy [0] ~2ho[1] 0 0 y(0] 
xf] | _ 4. | —Pi fl] Aoll+ rol2] —Aif] Ao l2] yl] 
x(2] 0 —ho [1] hy[0]— —Aof1] y[2] 
x[3] 0 2ho|2] —2hi[1] — ho|0] y[3] 


This concretely gives, in the case of order 4: 


x0] 1-10 0 y(0] 
el) o | ga 3-7 || vl 
x2] 0-1-4 y(2| 
x[3] 0-41 3 y[3] 


The synthesis equations to the order 2, in matrix notation: 


i = ( hi[0] — —2h8 [1] ) oT _ € 2) ce) 
xl] —2hi[1] ho [0] + 2h5 [2] ) \ yf] 11) \yf))° 


342 


5 Data Reduction: Lossy Compression 


Now return to our initial gradation. Write 


ToT. A dy! A i) al 
DAD P20 D222 <2 
3.3 3 3 3 3 3 3 
4444444 4 
Preis SG rs ay Oe oe 
6 6 6 6 6 6 6 6 
Ae SOE Me EGET SE 
8 8 8 8 8 8 8 8 
Then: 
1 1 1 1 0 
= 373 3 3 3 _ 0 
LL; = 5 5 5 5 HL; = 0 
7.25 7.25 7.25 7.25 0 
0 0 0 0 0 
_ 0 0 0 0 _ 0 
LH, = 0 0 0 0 HH; = 0 
0.5 0.5 0.5 0.5 0 
Apply now the 2D DWT to the order 4 to LLi. 
You have to evaluate the matrix product 
a 2 va 0 1 1 1 1 4 
See eae a See ull 
“8 ia. Be 5 5 5 5 = 
0 0 -5 5 7.25 7.25 7.25 7.25 0 
The result: 
1 0 1 0 
0 0 0 0 
6 0 6 0 
sO) gee 
This gives the four sub-bands 
Li, «wl 0 0 
LLe — HL2 = 
oe 00 
_ 0 0 _ 0 0 
LH2 = 9 9 HH2 = 0 0 


Finally, apply the 2D DWT to the order 2 to LLa: 


a 
2 
—i 

2 


1 1 1: 
2 11 23 
1) \ s9 39} | 1 2 
2 16 16 2 2 


We have obtained the following four sub-bands: 


LL3 = ~~, HL3 = 0, 
LH3= 8, HH3=0. 


Ale 


II 
a 
> | 
slags 
=) 


oooo o0cc°oe 


oo Co 2 oO Oo c'o 


oooo o0cocoeo 


Cole 


I ROO]OWR | 


Nie | Laxmi a} 
Nie 
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This ends the three level resolution of our given gradation. 

Now let us pass to the general situation. 

We dispose of a DWT (given by a filter bank). The ensuing D level, 2D DWT 
will function as follows: 

LLo = z[n] = (the component of) the digital image (considered as an N x N 
matrix, where N will be — for dichotomic convenience — a power of 2). 

When applying the 2D DWT of order N to LLo, we obtain 


LLi — a0 [n], HL = vor [n], 
LH, = y{))[n], HH = y{} [nl]. 


In interleaved version: 
1 
y [2na + bi, 2n2 + be] = ee [ni,n2], O<ni,ne< 5X —1. 
Now, we apply the 2D DWT of order 3N to LL, and we get: 


LL2 = ub, [n], HL2 = by [n], 
LH2 = yo [n], HH2 = WD [n]. 


We note 

We only have treated the positions of the interleaved matrix y [n] with coor- 
dinates (multiple of 2, multiple of 2). 

We replace these entries, while keeping the others unchanged, which gives the 
interleaved matrix y) [n] (it is still an N x N matrix), with 


1 
y [4n1 + 2b1,4n2 + 2b2] = ye?) [ni,n2], O<ni,ne< aN —1. 
Then, we apply the 2D DWT of order iN to Lhe, and we obtain: 


LL3 = Yao [n], HL3 = Yor [n], 
LH; = y{)[n], HHs = y$? (nl. 


We note 

We have only treated the positions of the interleaved matrix y [n] with coor- 
dinates (multiple of 4, multiple of 4). 

We replace these entries, while keeping the others unchanged, which gives the 
interleaved matrix y) [nj (this is still an N x N matrix) with 


1 
y [Bn + 4b1, 8n2 + 4b2] = ee [ni,n2], O<ni,n2e< ria -1 


and we continue. 
On the dth level, we have 


LLa = Yao [n], HLa = vor [nJ, 
LHa = y(n], HHa = y{? [nl], 


with 


= = 1 
y® [241 + 27-11, 242 + 27-1 do] = ae [ni,n2], O<ni,n2< gaN - 1. 
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Return to our example, in interleaved version. The considered component of the 


digital image will hence be given by 


DVD 1D De D2 PD 0) 
3.3 3 3 3 3 3 3 
4444444 4 
5 5 5 5 5 5 5 5 
6 6 6 6 6 6 6 6 


et Uke AG” Fhe ake 


8 8 8 8 8 8 8 8 


Look now at the three levels of the treatment, in interleaved notation: 


6:0: O10':67O'O. 1S OO GO C1000 
SCWMORMIN DODO DCO COHN COO O COO COUN 
oqoooco SOO OOS O29: +970 SOO O"S0 
Om OofsHIN HOO O8|IGOamin COO OS OO alain 
fess a a a a =m) Scio Soto o'6 lor SPSL SS 
Sas etn S6oe SSeS da CO OOS CORI 
3 ama sooo SoS Se CP Oore oro6 

HOO O8ISO alma BIBS SO ORISS alania 
O19 0 Asx IN aie s 


= = 
a S 


y 


> 
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Scheme of a D level, 2D DWT (in concrete applications, D = 4,5). 
LLo = A component of the 
digital image 


ine Rp: HL, Dth resolution 
LH, HH, level 

LL» Rp-1: HL (D — 1)th resolution 
LH» HH» level 

LLp_1 Ra: HLp-1 Second resolution 
LHp-1 HHp-1 level 

J 

LLp Ri: HLp First resolution 
LHp HHp level 

Ro 


Language The three (2D) sub-bands needed for the passage from the resolution 
LL, to the resolution LL,y—1 will be called the resolution levels Rp—,r+1. 


In order to get a first idea of the degree of influence which have the resolution 
levels for the faithful reconstruction of the image, start with 


105 0 
(Ro, R1) = #3 0 
32 


in our example of a three level resolution for a 8 x 8 gradation. 


In Re and R3, let us set LH2 and LH; to zero: the vertically high-pass informa- 
tion is “skipped”. 

In other words, we have to apply the synthesis transform to the interleaved 
matrix 


00000 0 0 
0 00 0 0 0 0 0 
0 00 0 0 0 0 0 
73) 9 0 0 0 0 0 0 0 
Be Ge OI 05 
0 00 0 0 0 0 0 
0 00 0 0 0 0 0 
0 00 0 0 0 0 0 
First, use the 2D synthesis transform to the order 2: 


; 1 -1 105 9 Te 1 1 
w= (2) CE 0) (4 t)-( @) 
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This gives — in interleaved version: 


(2) _ 


oo oBsoCoOCorR 
ocoococoooce 
oooocoocoooceo 
oc oes 'o 67S 
SCO ORBZOCOOR 
oooocooocooco 
oooocoocoooce 
O'S OO: OOOO 


Now, we have to apply the 2D synthesis transform of order 4 (to the entries on the 
(even, even) positions of the interleaved matrix). 


1 -1 0 O 102-10 1 5 O 0 
1 5 1 1 5 1 1 
a 0 0 0 af 8 st ot 
oa 2 4 2 4 4 2 2 
mi=lo -2 1 -t] | 2 0 8 0 qo f 4 4 
0 -; 1 2 0 0 0 0 0 -; -4 2 
dy “ile ‘vie Al 
105 105 105 105 
=— 2 2 2 2 
= 9 9 9 9 
a 38 38 gf 
16 16 16 16 
This gives — in interleaved version: 
1 0 1 0 1 0 1 +0 
0 0 0 0 0 0 0 0 
1 we (0) 1 aa 0 a 0 at 0 
for 0 0 0 0 0 0 0 0 
0% 0 f 0 % 0 
0 0 0 0 0 0 0 0 
0 § 0 G0 Fo 
0 0 0 0 0 0 0 0 
Finally, we reconstruct the image: multiply 
1 -1 0 0 0 0 0 0 1 01 0 1 0 1 «0 
3 3 3 -7 0 0 0 0 0 0 0 0 60 0 0 0 
OD Sy Oe De 08 ||| See ae on ae 0 
0 -t it 3 3 =a 0 0 Os Or Oe 0 Oe 10 60 
0 0 0 -$ 1 -fo o || 8 0 8 0 8 0 ¥ 0 
0 0 0 -%; § 3% % =} 0 0 0 0 0 0 0 0 
Ora? 0 Rt Sage Te ee 2s 0 8 0 2 0 8 0 
0 0 0 0 0-5 1 8 0 0 0 0 0 0 0 0 
1 01 0 1 0 1 =0 
a 0 st 0 ra 0 a 0 
105 0 105 0 105 0 105 0 
32 32 32 32 
283 0 283 0 283 0) 283 0 
=! &% 9 &% 9 8% 9g 8 9g 
16 48 48 16 
Be 804 See OP - B2s gt Ber 0 
1 48 48 16 
8 0 8 0 89 0 89 0 
8S 9 8 9 8 9 & G 
16 16 16 16 


Then, multiply 
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1 0 1 0 1 40 #1 
137 137 137 137 
7 7 a 7 a 71 
105 105, 105 105 
32 0 ay 9 ay O 35 
283 283 283 283 
7 a a a i 7 a a7 
89 89 89 89 
te 9 t¢ 9  O ff 
89 89 89 89 
te 9 t¢ 9  O FF 
89 89 89 89 
wt 9 t¢ 9  O 4 
89 89 89 89 
wt 9 i 9  O 4 
111 %21é%21é%721éi21éi421 
137 137 137 137 137 137 137 137 
4 64 64 64 64 64 64 64 
fos 105 fb 05 105 5 15 fe 
5s SH TR RR RR RR SB 
4 GF G4 G4 G4 GF G4 G4 
9 89 89 89 89 89 89 2&9 
ig 16 a6 is JG Ig a6 J 
89 89 89 89 89 89 B9 8 
88 88 88 88 88 88 88 88 
i is a6 is ig Ws a 3 
89 89 89 89 89 89 B89 8 
16 16 16 16 16 16 16 16 


oo oo co 090. oO oC Oo 


1 3 0 0 0 0 0 0 
-1 3 -3-;0 00 0 
Ore 176. (0) 0 Oy 10 
Ose aang eg eOr (OF I. 
000 4 1 4 0 0 
Or. Oey Gad 
00000 5 1 1 

ay ee 
00 0 0 0-4-3 3 


1 1 1 1 1 1 1 1 
2.14 2.14 2.14 2.14 2.14 2.14 2.14 2.14 
3.28 3.28 3.28 3.28 3.28 3.28 3.28 3.28 
4.42 4.42 4.42 4.42 4.42 4.42 4.42 4.42 
5.56 5.56 5.56 5.56 5.56 5.56 5.56 5.56 
5.56 5.56 5.56 5.56 5.56 5.56 5.56 5.56 
5.56 5.56 5.56 5.56 5.56 5.56 5.56 5.56 
5.56 5.56 5.56 5.56 5.56 5.56 5.56 5.56 


We note: when annihilating the vertically high-pass information for two resolution 
levels, we have decompressed into a half-constant gradation. 


Exercises 


The DWT of Cohen—Daubechies—Fauveau 


The DWT 9/7 CDF — which is, by the way, a constituent of the standard JPEG 
2000/1 — is defined by the following two analysis impulse responses: 


ho[—4 
ho[—3 
ho[-2 
hé{-1 

ho[0 
h\[-3 
hi [-2 
hi[-1 

hi [0 


= 0.026748757411, 
= —0.016864118443, 
= —0.078223266529, 
= 0.266864118443, 


= 0.602949018236, 


= ht 
=h} 
= ht 


4 
3 
2 
1 
0 
3 
2 


1 


= 0.045635881557, 
= —0.028771763114, 
= —0.295635881557, 


= 0.557543526229. 


Note that the coefficients are approximations of irrational expressions (cf. Exer- 
cise (7) at the end of the filter bank section). 


In the sequel, we shall consider the interleaved transforms y of matrix schemes 
x= y, in 16 x 16 format. The intervening filter banks will be the DWT 5/3 spline, 
the DWT 7/5 Burt and the DWT 9/7 CDF, respectively. 
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(1) For the three considered transforms, determine y™ in case where x = y is 


constant. 
(2) Find, for each of the three DWTs, the matrix scheme x 


yO such that: 


(3) Knowing that the gradation: 


1Tidtididtidtidtdidiiii 
DD SD DAD De a2 Dh Died Die 2: 2 Dt 22ND 


3.3 3 3 3 3 3 3 3 33 3 3: 3~°3~3 
4444444444444444 
5555 55 5 555 5 5 5 5 5 5 


6 6666 6 666 6 6 6 6 6 6 6 


ORE Eee he OO 
(oy 8 8:8 8.8 BB 8 88 8 Bes 8.8 8 


9999999999999 9 9 9 


x=y 


10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 
11111111 111111111111111111111111 


12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 


13 13 13 13 13 13 13 13 13 13 13 13 13:13 13 13 
14141414141414141414141414141414 


15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 


16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 
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is transformed into 


(0) 


(0) 


5.570 0 0 


0 


i) 
0 


0 
i) 


0 
4.570 0 0 


(4) — 


(a) y 


0 


0 


(0) 


0 0 02.280 0 0 0 


10) 


2.280 0 0 


1.130 0 01.130 0 01.130 0 01.130 0 0 


0.5 00.5 0 0.5 00.5 0 0.5 00.5 0 0.5 


00.5 0 


5.93 0 


—0.1 0 —0.1 0 


—0.1 0 —0.1 0 


0 —0.27 0 


—0.1 0 —0.1 0 


0 —0.27 0 


—0.1 0 —0.1 0 


0 


0 O —0.27 0 


(0) 


0 


—0.27 0 


—0.61 0 


[DWT 7/5 Burt], 


0 
0 


0 
0 


(0) 


(0) 


0 3.01 


3.01 


(0) 


1.49 0 
0.7 


0.7 O 0.7 0 0.7 0 0.7 0 0.7 0 0.7 0 


0 0.7 O 


(0) 


6.02 0 


0.12 0 0.12 0 0.12 0 0.12 0 0.12 0 0.12 0 0.12 0 0.12 0 


0.19 O 


0 


(0) 


(0) 


i) 
0 
0 


0 


0 


0 0.01 


0.01 


[DWT 9/7 CDF], 


0 
0 


0 
0 


0 


(0) 


0 —0.17 0 0 0 —0.17 0 0 oO —0.17 0 


(0) 


—0.17 0 


0 


1.93 0 


0) 


1.93 0 
—0.09 0 —0.09 0 —0.09 0 —0.09 0 —0.09 0 —0.09 0 —0.09 0 —0.09 0 


350 5 Data Reduction: Lossy Compression 


what will be the interleaved transformed matrix y of 


16 15 14 13 12 11 #10 9 8 7 65 43 2 «421 
16 15 14 13 #12 11 #10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 #10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 #10 9 8 7 65 43 2 «421 
16 15 14 13 #12 11 #10 9 8 7 65 43 2 «421 
16 15 14 138 12 11 #10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 #10 9 8 7 65 43 2 «421 
= ag) _ 16 15 14 13 12 11 10 9 8 7 65 4 3 2 «1 
16 15 14 13 12 11 #10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 #10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 #10 9 8 7 65 43 2 «421 
16 15 14 13 12 11 10 9 8 7 65 43 2 «41 
16 15 14 13 #12 11 #10 9 8 7 65 43 2 «421 


in each of the three cases? 


(4) Show that the value a = $ is an eigenvalue for the transformation 


(0) 


x= Roy a=y) 
y y-y 


for each of the three 2D analysis filter bank versions that we consider. (You will 
find a common eigenvector (i.e. a 16 x 16 matrix) according to the indication for 
Exercise (3) of the preceding section.) Will the value a = 4 also be an eigenvalue 
of the transform x = y —+ y = y“4)? 

(5) Consider the following four 16 x 16 matrix schemes: 


1010101010101010 1111111100000000 
0101010101010101 1111111100000000 
1010101010101010 1111111100000000 
0101010101010101 1111111100000000 
1010101010101010 1111111100000000 
0101010101010101 1111111100000000 
1010101010101010 1111111100000000 
ee ee es SO ee 
1010101010101010 Od000000011111111 
0101010101010101 0000000011111111 
1010101010101010 0000000011111111 
0101010101010101 0d000000011111111 
1010101010101010 0d000000011111111 
0101010101010101 0d000000011111111 
1010101010101010 0d000000011111111 
0101010101010101 0d000000011111111 


111111111111111 
111111111111111 
110000000000001 
110000000000001 
110000000000001 
110000000000001 
110000000000001 
110000000000001 
1 
1 
1 
lt 
1 
1 
1 
1 
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1 
1 
ak 
1 
1 
1. 
1 
1 
1 
a 
1 
1 
1 
1 
1 
1 


1100000000000000 
1110000000000000 
0111000000000000 
0011100000000000 
0001110000000000 
0000111000000000 
0000011100000000 
0000001110000000 


*3=110000000000001 *4“0000000111000000 
10000000000001 0000000011100000 
10000000000001 0000000001110000 
10000000000001 0000000000111000 
10000000000001 0000000000011100 
10000000000001 0000000000001110 
11111111111111 0000000000000111 
11111111111111 0000000000000011 

Then, we are given the eight matrix schemes yi, yo,..., Ys. 
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Each of these matrices is an interleaved version y“ of y = X1,X2,X3,X4 by 
either the DWT 5/3 spline or the DWT 7/5 Burt or the DWT 9/7 CDF (hence, 
there are 12 possibilities). 
Decide, with a minimum of pencil-and-paper effort, which is the decompressed 
version of each of the yi, 1 < i < 8 — and by which of the three DWTs at our 
disposal. 


Example y3 is the interleaved version y“) of x; by all the three transforms. 


~ _0.07 


0.24 0.1 


0.17 —0.16 0.05 0.04 
0.1 —0.13 0.07 —0.2 —0.05 0.04 0.02 
0.17 0.07 0.33 0.05 


0 0 —0.07 
0 0 
0 —0.06 —0.03 0.02 0.01 


—0.16 —0.2 0.05 —0.14 0.04 —0.22 —0.06 0.04 0.02 
0.04 0.29 0.04 —0.01 —0.06 0.01 


0.05 —0.05 


0.04 0.04 —0.06 —0.22 0.04 —0.14 0.04 —0.22 —0.06 0.04 0.02 


0 


0 0.02 —0.03 —0.06 —0.01 0.04 0.41 
0.02 
0.01 


0 i) 


i) 

0 0 
—0.01 O 
0 0 
0.05 0 
0 

1) 

1) 
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0500000000000 00 00 


0 0.50 0.5 00.5 0 0.5 0 0.5 0 0.5 0 0.5 0 0.5 
0 0000000000000 0 0 
0 0.50 0.5 00.5 0 0.5 0 0.5 0 0.5 0 0.5 0 0.5 
0000000000000 00 0 
0 0.50 0.5 00.5 0 0.5 0 0.5 0 0.5 0 0.5 0 0.5 
0000000000000 00 0 
7 0 0.50 0.5 00.5 0 0.5 00.5 0 0.5 0 0.5 0 0.5 
3 = 
0 0000000000000 0 0 
0 0.50 0.5 00.5 0 0.5 0 0.5 0 0.5 0 0.5 0 0.5 
0 0000000000000 0 0 
0 0.5 00.5 00.5 0 0.5 0 0.5 0 0.5 0 0.5 0 0.5 
0000000000000 0 0 0 
0 0.50 0.5 00.5 0 0.5 0 0.5 0 0.5 0 0.5 0 0.5 
000000000000 00 0 0 
0 0.50 0.5 00.5 0 0.5 0 0.5 0 0.5 0 0.5 0 0.5 
0.13 0.2 -0.19 0.06 0 0.06 0 0.13 0 0 0 018 0 0 0 
—0.13 0.09 —0.19 —0.09 0 0 0 0 0 0 ) 0 0 0 0 
0.09 0.41 0.09 —0.03 —0.09 0.03 0 0.03 0 0 0 0 0 0 0 
—0.19 0.09 —0.13 0.09 —0.19 —0.09 0 0 0 ) ) 0 0 0 0 
—0.09 —0.03 0.09 0.52 0.09 —0.03 —0.09 0 0 0.03 0 02 0 0 ) 
0 —0.09 —0.19 0.09 -0.13 0.09 —0.19 —0.09 0 0 0 0 0 0 ) 
0 0.03 —0.09 —0.03 0.09 0.44 0.09 —0.03 —0.09 0.04 0 003 0 -0.01 0 
0 0 0 -0.09 —0.19 0.09 —0.13 0.09 —0.19 -0.09 0 0 0 0 ) 
0 003 0 0 0.09 —0.03 0.09 0.45 0.09 —0.03 —0.09 -0.24 0 0.07. 0 
0 0 ) 0 0 0.09 —0.19 0.09 —0.13 0.09 —0.19 —0.09 0 0 ) 
0 0 0 0.03 6 0.04 —0.09 —0.03 0.09 0.44 0.09 0.01 —0.09 0.11 0 
0 0 0 0 ) 0 0 —0.09 —0.19 0.09 —0.13 0.09 —0.19 —0.09 0 
0 0 0 02 0 0.03 0 0.24 —0.09 0.01 0.09 0.53 0.09 —0.29 —0.13 
0 0 0 ) 0 0 0 0 0 0.09 —0.19 0.09 —0.13 0.06 —0.25 
0 0 0 0 0 -0.01 0 0.07 0 0.11 0.09 —0.29 0.06 0.55 —0.13 
0 0 0 0 ) 0 ) 0 0 0 0 -0.13 —0.25 -0.13 0 
—0.03 —0.06 0 0.15 0 0.01 0 -0.16 0 0 0.01 0.09 0.04 0.11 0 
0.05 0.19 0 0.23 0 0.23 0 0.23 0 0.23 0.01 0.25 0.06 0.05 0.01 
0.19 —0.06 0.01 —0.26 0 0.01 0 -0.240 0 0.04 0.2 —0.23 0.12 —0.03 
0 0.01 0 0.02 0 0.02 0 0.02 0 0.02 0 0.02 0 0 0 
0.23 —0.26 0.02 —0.07 0 0.04 0 —0.26 0 0.01 0.05 0.04 —0.28 0.5 —0.03 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0.23 0.01 0.02 0.04 0 0 0 0.04 0 0 0.05 0.03 —0.28 —0.02 —0.03 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0.23 —0.24 0.02 —0.26 0 0.04 0 —0.05 0 0.01 0.05 0.16 —0.28 0.47 —0.03 
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
0.23 0 0.02 0.01 0 0 0 0.01 0 0 0.04 0.01 ~0.27 0 0.03 
—0.01 0.04 0 0.05 0 0.05 0 0.05 0 0.04 0 0.05 0.01 0.01 0 
0.25 —0.2 0.02 0.04 0 0.03 0 0.16 0 0.01 0.05 0.03 -0.3 0.39 —0.04 
0.06 —0.23 0 —0.28 0 —0.28 0 —0.28 0 —0.27 0.01 0.3  —0.08 —0.06 —0.01 
0.05 0.12 0 0.5 0 -0.02 0 0.47 0 0 0.01 0.39 -0.06 —0.23 —0.01 
0.01 —0.03 0 —0.03 0 —0.03 0 —0.03 0 -0.03 0 —0.04 -—0.01 —0.01 0 
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(6) Around optimal quantization. 
Preliminary remark In lossy compression, there is a natural competition 
between two opposite criteria of quality: 


(a) The compression ratio. 
(b) The pictorial faithfulness of the decompressed image. 


The battle-field of these two criteria is the procedure of quantization. 
The compression ratio SE eee increases with the brutality of quantization. 

The pictorial faithfulness of the decompressed image will be guaranteed by del- 
icate quantization. 

We will have to find a satisfactory compromise. 

But let us first speak of the quantization operation. We have to associate with 
a real number y (between 0 and 1, say) an 8-bit byte (a modest proposal). The 
natural idea is to take the eight leading positions in the binary notation of y. The 
obvious dequantization will yield a rather satisfactory approximation of the initial 
value y. This procedure is legitimate, but too rigid for our needs. We aim for a kind 
of “arithmetical telescope” which carries out a weighting of the considered value y. 


Example Gradual quantization. 


(1) —0.31 +> (1, 01001111) 
The bit ‘1’ in the first position indicates the presence of a negative sign (the bit 
‘0’ would indicate its absence). 01001111 is the binary notation of the integer 
79 = [0.31 - 2°|1” 
In other words: 0.31 = 0.01001111* in binary notation (the unmasked part is 
clearly 2). 
The dequantization: (1,01001111) —+ —2.2 = —0.3105. (Dequantization “nat- 
urally” estimates the masked part of the quantized number: A real number 79.*« 
is generically 79.5). 

(2) —0.31 +> (1,00010011) 00010011 is the binary notation of the integer 19 = 


[0.31 - 2°| 
The dequantization: (1,00010011) + —32 = —0.3047. 

(3) —0.31 +> (1,00000100) 00000100 is the binary notation of the integer 4 = 
[0.31 - 24 


The dequantization: (1, 00000100) > — 4? = —0.28125. 

(4) —0.31 > (1, 00000001) 
00000001 is the binary notation of the integer 1 = |0.31- 27| 
The dequantization: (1,00000001) +> —4? = —0.375. 


Let us sum up. The progression (1), (2), (3), (4) shows a going down of interest: 
We are looking at the 8, 6, 4, 2 most significant positions in the binary notation of 
| y |= 0.31. The leading zeros serve as padding. 

For the intuitive. The more our quantization is brutal (and the dequantization 
imprecise), the more the ensuing compaction (of the whole of the quantized values) 
will be efficient — due to an important domination of the bit ‘0’ over the bit ‘1’. 


1” | | = the integer part of a (non-negative) real number. 
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Consequence Look now at the situation of an interleaved scheme y obtained 
from a 16 x 16 scheme x = y. 

Suppose that the value —0.31 there takes place at four different positions. Then 
the quantization will perhaps be carried out in four different ways, according to the 
importance that we have attributed to the considered position for the reconstruction 
of an acceptable image in synthesis. The more (the position of) this value will be 
important, the more significant bits of its binary notation will be taken into account. 

The objective of this exercise is thus to get a certain feeling for the weighting of 
the positions (or rather of the regions) of the interleaved scheme y. 


LLa| Hla 
Lal HAs Hts 
HL» 
LH; HH; 
HL, 
LH. A He 
LA, AM, 


We shall use the 2D DWT 5/3 spline. We choose as test matrix 


0 1.5 1.41.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 
0.1 0 1.5 1.41.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 
0.20.1 0 1.5 1.4 1.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 
0.30.20.1 0 1.5 1.41.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 
0.40.30.20.1 0 1.5 1.41.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 
0.5 0.40.3 0.20.1 0 1.5 1.41.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 
0.6 0.5 0.4 0.30.2 0.1 0 1.5 1.41.3 1.2 1.1 1.0 0.9 0.8 0.7 
0.7 0.6 0.5 0.40.3 0.20.1 0 1.5 1.41.3 1.2 1.1 1.0 0.9 0.8 
0.8 0.7 0.6 0.5 0.4 0.30.2 0.1 0 1.5 1.41.3 1.2 1.1 1.00.9 
0.9 0.8 0.7 0.6 0.5 0.40.3 0.20.1 0 1.5 1.41.3 1.2 1.1 1.0 
1.0 0.9 0.8 0.7 0.6 0.5 0.40.3 0.20.1 0 1.5 1.41.3 1.2 1.1 
1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.30.2 0.1 0 1.5 1.41.3 1.2 
1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.40.3 0.20.1 0 1.5 1.41.3 
1.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.20.1 0 1.5 1.4 
1.4 1.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.40.3 0.20.1 0 1.5 
1.5 1.4 1.3 1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 


xo yO = 
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This gives for the interleaved matrix!® 


0.727 0.1 0.305 —0O.1 0.405 0 —0.153 0 0.186 0 —0.002 0 —0.471 
—0.3 —0.3 0.2 —0.1 —0.05 0 0 0 0 0 0 0 0 
—0.383 —0.15 —0.283 0.2 0.27 —0.05 —0.166 0 —0.077 0 —0.002 1) —0.001 
1) 0 —0.15 —0.3 0.2 —0.1 —0.05 0 0 0 0 i) 0 
—0.411 1) —0.219 —0.15 —0.197 0.2 0.271 —0.05 0.227 i) —0.077 0 —0.278 
1) i) i) 0 —0.15 —0.3 0.2 —0.1 —0.05 10) 0 0 1) 
0.033 0 0.033 i) —0.211 —0.15 —0.267 0.2 0.272 —0.05 —0.164 0 —0.077 
(4) — 0 0 0 0 0 0 —0.15 —0.3 0.2 —0.1 —0.05 0 0 
y ~ —O.1 10) 0.016 0 —0.231 0 —0.211 —0.15 —0.147 0.2 0.272 —0.05 0.26 
0 0 i) i) 0 0 0 0 —0.15 —0.3 0.2 —0.1 —0.05 
0 1) i) i) 0.016 0 0.033 0 —0.211 —0.15 —0.267 0.2 0.213 
0 i) i) 0 i) i) 0 0 0 0 —0.15 —0.3 0.2 
0.334 0 0 0 0.114 0 0.016 0 —0.103 1) —0.199 —0.15 —0.186 
0 0 i) i) ie) 0 0 0 0 0 10) 1) —0.15 
0.113 10) 0 0 0.113 i) 0 0 0.141 0 0.056 0 
0.05 0 0.05 0 0.05 0 0.05 0 0.05 1) 0.05 0 0.050 


—0.113 —0.5 


1) 

0 i) i) 

0 0 —0.05 
i) 0 i) 

i) —0.114 —0.05 
0) i) 0 

i) —0.003 —0.05 
i) i) ie) 

0 —0.242 —0.05 
i) i) 0 
—0.05 —0.281 —0.05 

—0.1 —0.05 0 
0.2 0.413 —0.15 
—0.3 0.15 —0.2 


—0.156 —0.15 —0.294 0.45 


0 —0.15 —0.4 


In order to start naturally, let us first carry out a uniform quantization which 
saves the eight most significant positions in the binary notation of each of the 256 


coefficients of y: 


186 25 78 —25 103 0 -39 0 47 O 0 Q -120 0 —28 —12 
76 —76 51 25-12 0 0 0 0 0 0 0 0 0 0 0 
—98 —38 —72 51 69 —12-42 0 -19 O 0 0 0 0 0 -12 
0 0 38 —76 51 —25 —-12 0 0 0 0 0 0 0 0 0 
—105 0 -—56 —38—-50 51 69 —-12 58 O 19 0 71 #O 29 -—12 
0 0 0 0 38 —76 51 25-12 0 0 0 0 0 0 0 
8 0 8 0 —54 —38 —68 51 69 —12—-42 0 -—-19 0O 0 -12 
(4)_ 0 0 0 0 0 0 38 —76 51 —25 —-12 0 0 0 0 0 
a" ~25 0 4 0 59 0 54 —38 —37 51 69 —12 66 0 -62 —12 
0 0 0 0 0 0 0 0 38 —76 51 —25 —12 0 0 0 
0 0 0 0 4 0 8 0 -—54 —-38 -—68 51 54 -—12 —72 —-12 
0 0 0 0 0 0 0 0 0 0 -38 —-76 51 -25-12 0 
85 0 0 0 29 O 4 0 26 0 51 —38 —47 51 105 —38 
0 0 0 0 0 0 0 0 0 0 0 0 -38 —76 38 —51 
28 0 0 0 28 O 0 0 36 O 14 O —40 —38 —75 115 
12 0 12 0 12 0 12 0 12 0 12 0 12 0 -—38 —102 
The first row of q” begins as follows: 10111010, 00011001, 01001110, etc. 
Now the dequantized matrix: 
0.729 0.1 0.307 -0.1 0.404 10) —0.154 0 0.186 10) 10) 10) —0.471 0 0.111 
—0.299 —0.299 0.201 —0.1 —0.049 10) 10) 0 10) 0 0 i) 0 1) i) 
—0.385 —0.15 —0.283 0.201 0.271 —0.049 —0.166 10) —0.076 10) 10) 10) 10) 10) 10) 

0 0 —0.15 —0.299 0.201 —0.1 —0.049 0 10) 10) 10) i) 0 0 1) 
—0.412 0 —0.221 —0.15 —0.197 0.201 0.271 —0.049 0.229 10) —0.076 10) —0.279 10) —0.115 
0 0) 10) 10) —0.15 —0.299 0.201 —0.1 —0.049 10) 10) 10) 10) 10) 0 
0.033 0 0.033 10) —0.213 —0.15 —0.268 0.201 0.271 —0.049 —0.166 10) —0.076 10) 10) 

2(4) = 0 0 0 10) 10) 10) —0.15 —0.299 0.201 -—0.1 —0.049 10) 10) 0 10) 
~ —0.1 0 0.018 10) —0.232 10) —0.213 —0.15 —0.146 0.201 0.271 —0.049 0.26 0 —0.244 
0 10) 0 10) 1) 0 10) 10) —0.15 —0.299 0.201 —0.1 —0.049 10) 10) 
1) 0 0 10) 0.018 0 0.033 10) —0.213 —0.15 —0.268 0.201 0.213 —0.049 —0.283 
0 10) 0 10) 10) 0 10) 10) 10) 10) —0.15 —0.299 0.201 -—0.1 —0.049 
0.334 0 0 10) 0.115 10) 0.018 10) —0.104 10) —0.201 —0.15 —0.186 0.201 0.412 
1) 0 0 0 10) 1) 0 10) 10) 10) 10) 10) —0.15 —0.299 0.15 
0.111 10) 0 10) 0.111 10) 10) 10) 0.143 10) 0.057 0) —0.158 —0.15 —0.295 
0.049 10) 0.049 10) 0.049 0 0.049 10) 0.049 10) 0.049 i) 0.049 10) —0.15 


18 With duly rounded coefficients. 


—0.49 
0 
—0.049 
i) 
—0.049 
0 
—0.049 
0 
—0.049 
i) 
—0.049 
0 
—0.15 
—0.201 


0.451 
—0.4 
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We note that the maximum deviation between (the rounded version of) y and 
(the rounded version of) z“*) is 0.003 (a coefficient with value —0.003 becomes 0). 
Finally, apply the synthesis transform of the (2D) DWT 5/3 spline to 2419, 


-002 
-101 
-201 
-303 
-405 
-505 
-604 
-703 
-803 
-903 
002 
-102 
-202 
-301 
1.4 

1.498 


2(0) — 


PRREROOOOCCOOCSO 


1.5 1 


099 


-402 
-502 
603 
703 


903 
002 
102 
-201 
1.3 
1.398 1 


BRREOOCOCOOCCCOOO 


199 0. 
-302 0. 
0.3 0.197 
0.401 0.299 
0.502 0.4 
0.603 0.502 
803 0. 
ie) 
0 
1 
1 


404 1.3 
-004 1. 


503 1.404 
0 1.502 
098 0.004 
199 0.097 


703 0.602 


-803 0.703 
-903 0.802 
-002 0.901 
-101 1.001 
1.2 1.1 
-298 1.197 


1.196 1.096 
1.301 1.2 
1.401 1.303 
1.501 1.405 
0 1.501 
0.096 0.002 
0.197 0.095 
0.298 0.196 
0.4 0.299 
0.501 0.401 
0.603 0.502 
0.702 0.602 
0.801 0.702 
0.9 0.802 
dl 0.902 
1.097 1 


-896 
-999 
-101 
-204 
-304 


ORRRRROO 


-797 
-899 
002 
-102 
-203 
.306 


OrPrRRREHOO 


698 
801 


COORRRRRFHROOO 
wo 
=) 
iy 


0.598 0.399 0.3 


0.703 
0.808 
0.906 
1.003 
1.1 
1.197 
1.302 
1.404 
1.502 
0 
0.098 
0.2 
0.301 
0.402 
0.499 


DIBA 


0. 
0. 
0. 
0. 


-902 
-003 
-103 
202 
-305 
404 
505 
005 
103 
-204 
-301 


COSCORREREHEEHO 


0.4 
499 


roosos 
oO 
oO 
T 


0.198 


0.201 
0.299 
0.398 
0.496 
0.594 
0.702 
0.81 
0.904 
0.998 
1.096 
1.195 
1.303 
1.407 
1.504 
0 
0.099 


0.499 
0.601 
0.704 
0.803 
0.902 
1.001 
1.1 
1.203 
1.305 
1.407 
1.503 
0.006 
0.101 
0.201 
0.303 
0.4 


We note: The maximum deviation max{| y [i,j] — 2 [i,j] | 0 < i,j < 
15} = 0.012.2° What is the consequence for the faithfulness of the compres- 
sion/decompression? Clearly, everything depends on the precision with which x = 
yO transcribes the values of the digital image. Suppose that we started with the 
256 luminance sample values (in 8-bit bytes) of a 16 x 16 pixel region. The matrix 
x= yO will then be a “renormalization” of these values. Let us adopt the primitive 
viewpoint of a correspondence 


LLo 
LL, 


LL2 
LL3 


LL 


LH3 
Ri: 
LH4 


AL, 
AA, 
HL2 
HH» 
AL: 
HH3 
AL, 
HH4 


1.5 


0.3 


— 150 


— > 30 —— 


10010110 


00011110 


Then the luminance matrix derived from 2) differs from the initial luminance 
matrix in 31 octets (count the critical positions!). All that for a first orientation. 
But let us begin with the statement of the exercise. We shall uniformly quantize on 
each resolution level. 


We shall allocate to every resolution level R; a quantization factor Ni = bi 
(¢ = 0,1, 2, 3,4), which fixes the number 0; of the significant binary positions which 
we shall take into account — according to the quantization/dequantization procedure 
already discussed. 
Hence let y = sgn(y) - |y| be a coefficient of level R,. 


Quantization: q = ||y| - Ni] 


19 Tt is the q which undergoes entropy coding; hence the decompression recovers 
precisely q™ — its version to synthesize will be 24), 


20 |) y) — 2) |= 0.049. The matrix y — 2 “is worth 4 


in quadratic mean. 


+0.003 per coefficient” — 
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Dequantization: y’ = sgn(y) - go 
The value 0 will be quantized into 0 and dequantized into 0. The four coeffi- 
cients which determine Ro = LL as well as Ri = (LH1, HL4, HH) will be 
“optimally” quantized: No = N; = 2°. 
Philosophy: Do not touch the most primitive fragments of the image! 


Concerning the other resolution levels, we shall vary. 


(a) Choose the most brutal quantization: No = Nz; = Ny = 1 = 2°. Determine 
the dequantized scheme 2), then the decompressed scheme 0), Compare 20) 
with y: Maximum deviation max{| y [i,j] — 2© [i,j] | 0 < i,7 < 15} and 
Euclidean distance || y — 2© ||? 

(b) Now, let us widen our horizon of interest for the details: 

First step: No De: N3 = Na=1 2°. 

Second step: No = N3 = 2°, Ny = 1 = 2°.7+ 
Determine, in every case, the dequantized scheme 2), then its decompressed 
scheme 2°. Compare 2 with y©: Maximum deviation max{| yi, j] — 
2 [i,j] |0 < i,j < 15} and Euclidean distance || y© — z© ||? 

(c) Now pass to a graduated quantization: No = 2°, N3 = 2°, Na = 2%. This 
seems to be natural in two aspects: On the one hand, it is logical for efficient 
compression — it creates the greatest number of zeros in regions with many 
coefficients. On the other hand, it is logical for faithful image reconstruction 
— it is delicate where the numerical details appear in a “small team” and it is 
more brutal where the numerical details appear in a “big team”. 

Determine, as before, the dequantized scheme z“), then its decompressed 
scheme 2. Compare, once more, 2 with y: Maximum deviation max{| 
yO [i, j] — 2 [i,j] | 0 < 4,7 < 15} and Euclidean distance || y — 2© ||? 

(d) A last attempt, in order to verify if our viewpoint is right. Now invert the order 
of our quantization factors: N2 = 27, N3 = 24, Ny = 2°. Once more: Compute 
the dequantized scheme 2), then the decompressed scheme 20), Compare, for 
a last time, 2 with y: Maximum deviation max{| y© [i,j] — 2© [i,j] | 0 < 
i, j < 15} and Euclidean distance || y — 2 || ? 


Remark In practice, after long, honest and complicated considerations, we shall have 
a tendency to more or less resign in applying the following primitive formula: 
We shall fix No (which gives the maximum precision) and then put N; = No-2~‘. 


[The results (we work with the 2D DWT 5/3 spline): 


21 The third step has just been discussed.. 
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ooo 
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z(0) — 


Soo oooool SOS 9.099 


annananaa 
COOCOOCOCOO 
cococoono; 
222020900 


ooooossssssoooos 


-662 
-656 
-649 
-642 
-636 


629 
623 
616 
609 
609 
609 
609 
609 
609 
609 
609 


ecooCDDDOCDOCCCOCCOSD 


-745 
-729 
714 
-698 
-682 
-666 
651 
-635 
.619 
.619 
-619 
-619 
-619 
-619 
-619 
-619 
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-828 
-803 
-778 
-753 
.729 
.704 
.679 
-654 
-629 
-629 
-629 
-629 
-629 
-629 
629 
-629 


eco0DgpDD ODDO OCCCOCCOSO 


Hence max{| y [i, j] — 


7,682.77 


-911 
877 
843 
809 
775, 
741 
707 
673 
639 
639 
639 
639 
639 
639 
639 
-639 


eoooosssssssoosos 


es99090900000000000 


994 
951 
-908 
-865 
821 
-778 
«735 
-692 
-648 
-648 
-648 
-648 
-648 


648 
648 


-648 


SCODDDOCDOOCOCCOCOCOOHH 


O77 
025 
972 
-920 
-868 
-815 
-763 
-711 
-658 
-658 
658 
-658 
658 
-658 
658 
658 


1.16 

099 
037 
976 
914 
853 
791 
729 
668 
668 
668 
668 
-668 
-668 
-668 
-668 


eoopgpgCDCOCOCOCCCOOBRE 


1.16 

-099 
-037 
-976 
914 
-853 
-791 
.729 
-668 
668 
668 
-668 
668 
668 
668 
668 


essogpg0g09CCCCCoOrRRF 


1.16 1.16 
1.099 1.099 
1.037 1.037 
0.976 0.976 
0.914 0.914 
0.853 0.853 
0.791 0.791 
0.729 0.729 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 


1.16 

099 
037 
976 
914 
853 
791 
729 
668 
668 
668 
668 
668 
668 
668 
668 
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1.16 

099 
037 
.976 
.914 
-853 
-791 
-729 
-668 
668 
-668 
.668 
-668 
-668 
668 
.668 


eoogpCCCOCOCCCOOBF 


2006, i] | O < i,7 < 15} = 1.06 and || y 
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1.16 1.16 
1.099 1.099 
1.037 1.037 
0.976 0.976 
0.914 0.914 
0.853 0.853 
0.791 0.791 
0.729 0.729 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
0.668 0.668 
(0) jy 
eon) |= 


It is clear that there is no hope to reconstruct the cyclic structure of x = y 
by means of the four coefficients of LL3. 
(b) Actually, it is the “vicious” (highly non-pictorial) structure of x = y) which 
demands a thorough participation of the (regions of) details in an — even ap- 
proximate — reconstruction of the initial scheme. 
This is shown in the frustrating results of this part of the exercise. We see that 
the total suppression of the highest resolution level is — at least in our case — 
completely intolerable — and still less the simultaneous annihilation of the two 


highest resolution levels. 
But let us go on to the results. 


Begin with the first step quantization 


0.307 
0.263 
0.22 
0.177 
0.134 
0.305 
0.477 
0.649 
29) = 0.82 
0.93 
.039 
.148 
1258 
258 
258 
258 


BRR RRR 


0.66 
0.548 
0.436 
0.325 
0.213 
0.329 
0.446 
0.562 
0.679 
0.799 
0.919 

1.04 

1.16 

1.16 

1.16 

1.16 


-013 
-833 


oeooo0coooOoOH 
wo 
a 
i) 


1.366 
1.118 
0.869 
0.620 
0.371 
0.377 
0.384 
0.39 
0.396 
0.538 
0.68 
0.822 
0.964 
0.964 
0.964 
0.964 


1.72 1. 


1.402 
1.085 
0.768 
0.45 
401 
353 
304 
255 
408 
561 
713 
866 
866 
866 
866 


OOS SO: OOO. 


This yields max{| yO lt, j] — 
z°) ||= 5.157. 


22 So, we get, in quadratic mean, a deviation of J 
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A471 
283 
095 
907 
719 
615, 
512 
408 
305 


-411 


517 


-623 
.729 
.729 
.729 
.729 


SOCDOSDDDDCDCOCOCORHHEH 


+223 
.164 
-105 
-046 
-987 
.829 
671 
513 
.355 
-414 


474 


533 
592 
-592 
-592 
-592 


0.975 
1.045 
1.115 
1.186 
1.256 
1.043 
0.83 
0.618 
0.405 
0.418 
0.43 
0.443 
0.455 
0.455 
0.455 
0.455 


0.727 
0.926 
1.125 
1.325 
1.524 
1.257 
0.99 
722 
455 
421 
387 
353 
318 
318 
318 
318 


OOO OOO OPO O 


631 
803 
976 
148 
320 
166 
O11 
-856 
701 
621 
0.54 
0.459 
0.379 
0.379 
0.379 
0.379 


COORFRFRROOO 


0.535 0.439 0.344 0.344 0.344 0.344 
0.68 0.558 0.435 0.435 0.435 0.435 
0.826 0.676 0.526 0.526 0.526 0.526 
0.971 0.794 0.617 0.617 0.617 0.617 
1.116 0.912 0.708 0.708 0.708 0.708 
1.074 0.982 0.891 0.891 0.891 0.891 
1.032 1.053 1.074 1.074 1.074 1.074 
0.99 1.123 1.257 1.257 1.257 1.257 
0.947 1.193 1.439 1.439 1.439 1.439 
0.82 1.02 1.22 1.22 1.22 1.22 
0.693 0.847 1 1 1 1 
0.566 0.673 0.78 0.78 0.78 0.78 
0.439 0.5 0.561 0.561 0.561 0.561 
0.439 0.5 0.561 0.561 0.561 0.561 
0.439 0.5 0.561 0.561 0.561 0.561 
0.439 0.5 0.561 0.561 0.561 0.561 
and || yO — 


214, 3] | 0 < i,7 < 15} = 0.939 


that the Euclidean norm of the constant matrix of value 1 is equal to 16). 
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Now the second step quantization 


0.505 0.383 0.262 
0.604 0.502 0.401 


0.173 2 0.951 1.190 1.430 1.259 1.088 0.995 2 0.790 0.677 0.677 


0.102 0.953 1.804 1.476 1.147 2 0.996 0.896 0.797 0.698 0.598 0.499 0.399 0.288 0.177 0.177 
0.76 0.545 1.015 1.214 1.413 9 1.086 0.993 0.899 0.801 0.703 0.601 5 0.387 0.275 0.275 
0.05 0.138 0.226 0.952 1.678 7 1.176 1.089 1.002 0.905 0.808 0.704 6 0.487 0.373 0.373 
0.228 0.201 0.175 0.563 0.951 9 1.427 1.259 1.09 0.998 0.906 0.803 .7 0.586 0.472 0.472 
0.405 0.264 0.124 0.174 0.225 8 0.685 0.57 0.57 
0.217 0 
0.2 0. 


-261 0.121 


i) 
1) 
0 
2 1.678 1.429 1.179 1.091 1.003 0.902 0. 
9 
003 0.894 0.785 0.785 
0 


1.07: 
1.24) 
1.42 
1.18 
0.95 
0.56 i) 
0.172 0.223 0.952 1.681 1.427 1.173 1.088 1. 
2(0) — 0.703 0.603 0.502 0.381 0.26 0.217 0.173 0.564 0.955 1.191 1.427 1.259 1.091 0.985 0.879 0.879 
0.803 0.703 0.603 0.502 0.4 0.262 0.123 0.176 0.229 0.955 1.68 1.429 1.178 1.076 0.973 0.973 
0.903 0.803 0.703 0.602 0.501 0.382 0.262 0.219 0.177 0.565 0.953 1.191 1.430 1.245 1.06 1.06 
1.002 0.903 0.803 0.703 0.603 0.50 
1.102 1.002 0.903 0.802 0.702 0.60 
1.202 1.102 1.002 0.901 0.801 0.70 
1.313 1.213 1.113 1.013 0.913 0.81 
1.425 1.325 1.224 1.124 1.024 0.92 
1.425 1.325 1.224 1.124 1.024 0.92 


2 0.401 0.263 0.124 0.174 0.225 0.953 1.681 1.414 1.147 1.147 
2 0.502 0.382 0.262 0.218 0.174 0.565 0.956 1.173 1.39 1.39 
2 0.603 0.502 0.401 0.263 0.124 0.177 0.23 0.932 1.633 1.633 
4 0.716 0.615 0.515 0.395 0.275 0.233 0.191 0.598 1.004 1.004 
6 0.829 0.729 0.629 0.528 0.426 0.289 0.152 0.264 0.375 0.375 
6 0.829 0.729 0.629 0.528 0.426 0.289 0.152 0.264 0.375 0.375 
with max{| y [i, j] — 2 [6,7] | 0 < i,j < 15} =1.125 and || yO — 2© |= 
3.165. 

We note here a disaster of the maximum deviation, which is rather 
impressive — and greater than at the first step. But it comes up at the boundary, 
and in a corner of great numerical variation. We note that the annihilation of 
the highest resolution level gives rise to a hook ‘|’ of two identical rows/columns 
at the boundary south — boundary east of the reconstructed scheme (we re- 
mark that the annihilation of the two highest resolution levels, at the first 
quantization step, has created a hook of four identical rows/columns at the 
boundary south — boundary east of 2) (and the annihilation of the three 
highest resolution levels has created a hook of eight identical rows/columns — 


cf. (a)). 


186 1 9 —-1 25 0-4 0 47 0 0 O —30 0 —3 0 
-4-4 3 -1 0 00 0 0 00 0 0 0 0 0 
—-12-2-9 3 8 0-50 -2 00 0 0 0 0 0 
0 0-2-4 3 -10 0 0 0 0 0 0 0 0 0 
—26 0 —7-2-12 3 8 O 14 0 —-2 0 -17 0 —-3 0 
0 0 0 0 -2-4 3 -1 0 0 0 0 0 0 0 0 
1 0 1 0 -6 -2-8 3 8 0-50 -2 0 0 0 
ea 0 0 0 0 0 0-2-4 3 -10 0 0 0 0 0 
—25 0 0 0 —-14 0 -6-2-37 3 8 0 16 0 —7 0 
0 0 0 0 0 0 0 0 -2-4 3 -1 0 0 0 0 
0 0 0 0 0 0 1 0 -6 -2-8 3 6 0 -9 0 
0 0 0 0 0 0 0 0 0 0-2-4 3 -10 0 
21 0 00 7 0 0 0 -6 0 -6—-2-11 3 13 —-2 
0 00 0 0 00 0 0 0 0 0 —2 -4 2 -3 
3 0 0 0 3 0 0 0 4 0 1 0 -5 -2-9 7 
0 00 0 0 00 0 0 0 0 0 0 0 —2-6 


z(0) 
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0.02 1.463 1.406 1.271 1.137 1.084 1.031 0.916 0.801 0.697 0.594 0.49 0.387 0.277 0.168 0.168 
0.125 0.042 1.490 1.444 1.336 1.26 1.09 0.997 0.903 0.802 0.7 0.595 0.489 0.38 0.271 0.271 
0.23 0.059 —0.051 1.461 1.348 1.342 1.149 1.078 1.006 0.906 0.807 0.699 0.592 0.482 0.373 0.373 
0.32 0.163 0.037 0.028 1.488 1.491 1.337 1.256 1.081 0.993 0.905 0.8 0.694 0.585 0.476 0.476 
0.41 0.299 0.188 0.064 0.004 1.483 1.338 1.341 1.156 1.08 1.004 0.9 0.797 0.688 0.578 0.578 
0.512 0.41 0.309 0.18 0.083 0.05 1.485 1.491 1.341 1.247 1.06 0.978 0.896 0.793 0.689 0.689 
0.613 0.522 0.431 0.328 0.225 0.085 0.008 1.485 1.338 1.321 1.116 1.056 0.996 0.898 0.801 0.801 
0.715 0.608 0.501 0.412 0.323 0.188 0.85 0.05 1.483 1.478 1.317 1.240 1.068 0.982 0.896 0.896 
0.816 0.694 0.572 0.497 0.422 0.323 0.225 0.083 0.004 1.479 1.330 1.329 1.141 1.066 0.992 0.992 
0.912 0.792 0.672 0.587 0.502 0.414 0.326 0.187 0.079 0.044 1.479 1.483 1.331 1.233 1.042 1.042 
1.008 0.890 0.771 0.677 0.582 0.504 0.427 0.322 0.217 0.078 0.002 1.480 1.334 1.307 1.092 1.092 
1.104 0.999 0.895 0.802 0.709 0.605 0.5 0.406 0.312 0.18 0.08 0.048 1.485 1.495 1.349 1.302 
1.199 1.108 1.018 0.927 0.836 0.705 0.574 0.49 0.406 0.313 0.221 0.085 0.012 1.527 1.418 1.324 
1.309 1.218 1.127 1.036 0.945 0.816 0.688 0.605 0.523 0.415 0.307 0.183 0.09 0.074 1.527 1.387 
1.418 1.327 1.236 1.146 1.055 0.928 0.801 0.721 0.641 0.517 0.393 0.312 0.23 0.09 0.012 1.574 
1.418 1.327 1.236 1.146 1.055 0.928 0.801 0.721 0.641 0.517 0.393 0.312 0.23 0.137 0.105 0.043 


Here max{| y [é, j] — 2 (4, J], | 0 < 4,7 < 15} = 0.108 and || yO —2© |= 
0.613. In quadratic mean, the deviation per position is +0.04. But let us look 
more closely at the matrix q™. If we write the absolute values of the entries 
in 8-bit bytes, we obtain 1,872 zeros per 2,048 bits. In the language of the 
first chapter (on compaction) we thus have: po = 0.914. The entropy of the 
considered binary source: H(p) = 0.423. 

Arithmetic coding will reduce to approximately 43%. This is satisfactory, since 
the scheme x = y is pictorially chaotic. 

There remains a little uneasiness concerning the low precision of the reconstruc- 
tion. We shall try to find a remedy by a treatment which takes into account the 
irregularities at the boundaries coming from the symmetric extensions of the 
considered row and column vectors. 

More precisely, we shall try to maintain a “frame” of 2/3/4 boundary 
rows/columns where the quantization will be as clement as possible; this 
will lower the compression ratio, but will — hopefully — increase the pictorial 
faithfulness of the reconstructed scheme. For a brief information, let us look 


quickly at the variant where the two boundary rows/columns will be quantized 
by No = 2°. We obtain 


—0.01 1.504 1.423 1.308 1.192 1.1 1.008 0.905 0.802 0.7 0.598 0.496 0.394 0.294 0.195 0.098 
0.108 0.003 1.494 1.401 1.286 1.226 1.073 0.988 0.903 0.803 0.703 0.6 0.496 0.396 0.296 0.198 
0.227 0.093 —0.037 1.488 1.376 1.351 1.138 1.071 1.005 0.907 0.809 0.704 0.598 0.497 0.396 0.298 
0.32 0.182 0.046 0.044 1.505 1.497 1.332 1.252 1.079 0.993 0.907 0.804 0.7 0.598 0.496 0.399 
0.414 0.303 0.192 0.069 0.008 1.486 1.339 1.340 1.153 1.079 1.005 0.904 0.803 0.7 0.597 0.499 
0.503 0.404 0.306 0.181 0.086 0.052 1.486 1.49 1.339 1.247 1.062 0.983 0.903 0.803 0.703 0.606 
0.591 0.506 0.421 0.324 0.227 0.086 0.008 1.485 1.336 1.321 1.119 1.061 1.003 0.906 0.810 0.712 
_ 0.707 0.603 0.498 0.411 0.324 0.189 0.085 0.05 1.483 1.48 1.32 1.245 1.076 0.988 0.901 0.803 
0.823 0.699 0.576 0.499 0.422 0.323 0.225 0.083 0.004 1.482 1.334 1.335 1.149 1.070 0.992 0.894 
0.916 0.795 0.673 0.587 0.501 0.413 0.326 0.187 0.079 0.044 1.477 1.482 1.332 1.249 1.073 0.975 
1.009 0.890 0.771 0.676 0.581 0.503 0.426 0.322 0.218 0.074 —0.006 1.473 1.327 1.334 1.154 1.056 
1.102 0.997 0.892 0.8 0.707 0.603 0.5 0.406 0.313 0.177 0.073 0.044 1.484 1.461 1.281 1.182 
1.194 1.104 1.014 0.923 0.833 0.703 0.573 0.491 0.408 0.311 0.214 0.084 0.016 1.529 1.416 1.316 
1.293 1.203 1.113 1.022 0.932 0.803 0.674 0.592 0.51 0.41 0.31 0.183 0.088 0.065 1.499 1.397 
1.393 1.302 1.212 1.122 1.031 0.903 0.775 0.693 0.612 0.509 0.405 0.314 0.223 0.1 0.016 1.52 
1.49 1.4 1.31 1.219 1.129 1.001 0.872 0.791 0.709 0.606 0.503 0.412 0.321 0.199 0.116 0.018 


with max{| y© [é, jJ—2© [i, j] |0 < i,j < 15} =0.097 and || yO —z ||= 0.4. 
Are we satisfied with this result? We shall have a modest compression rate for a 
sensible loss of information: 17 positions (per 256) of the initial scheme will not 
be correctly restored if one rounds roughly to the first decimal position. But our 
test scheme is pictorially very “excited”, and the 16 x 16 format is too small in 
order to permit an efficient taming of the digital reflux from the boundaries, due 
to symmetric extension, whilst aiming at the same time at a high compression 
ratio. 
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0.238 1.61 1.357 1.167 0.977 0.985 0.994 1.03 1.066 1.055 1.043 1.031 1.02 0.939 0.859 0.75 
0.172 0.011 1.467 1.334 1.216 1.154 1.099 1.032 0.964 0.924 0.884 0.852 0.819 0.739 0.659 0.55 
0.105 0.052 —0.017 1.524 1.472 1.330 1.205 1.033 0.861 0.793 0.725 0.672 0.619 0.539 0.459 0.35 
0.09 0.106 0.115 0.09 1.699 1.477 1.28 1.038 0.805 0.693 0.581 0.5 0.419 0.339 0.259 0.149 
0.074 0.153 0.231 0.29 0.332 1.648 1.37 1.051 0.748 0.593 0.438 0.328 0.219 0.139 0.059 —0.051 
0.203 0.236 0.27 0.354 0.43 0.142 1.486 1.205 0.948 0.818 0.695 0.577 0.458 0.382 0.306 0.196 
0.332 0.32 0.308 0.41 0.513 0.268 0.008 1.383 1.164 1.051 0.953 0.825 0.697 0.625 0.553 0.443 
7(0)= 0.461 0.461 0.461 0.518 0.575 0.374 0.165 —0.064 1.339 1.248 1.181 1.078 0.982 0.907 0.831 0.722 
0.59 0.602 0.613 0.625 0.637 0.472 0.307 0.121 —0.08 1.469 1.424 1.338 1.268 1.188 1.109 1 
0.766 0.745 0.724 0.703 0.682 0.507 0.332 0.208 0.076 0.038 1.633 1.603 1.596 1.508 1.429 1.319 
0.941 0.888 0.834 0.78 0.727 0.542 0.357 0.287 0.217 0.24 0.248 1.891 1.939 1.836 1.748 1.639 
1.117 1.031 0.944 0.858 0.771 0.634 0.497 0.417 0.337 0.421 0.498 0.554 2.242 2.125 2.031 1.926 
1.293 1.174 1.055 0.936 0.816 0.727 0.637 0.547 0.457 0.595 0.732 0.85 0.951 2.438 2.33 2.229 
1.373 1.254 1.135 1.016 0.896 0.822 0.748 0.674 0.6 0.676 0.753 0.919 1.078 0.959 2.473 2.383 
1.453 1.334 1.215 1.096 0.977 0.918 0.859 0.801 0.742 0.758 0.773 0.981 1.189 1.113 1.021 2.506 
1.563 1.443 1.324 1.205 1.086 1.027 0.969 0.91 0.852 0.867 0.883 1.091 1.299 1.219 1.123 1.014 


with max{| y© [é, j] — 2© [é, j] | 0 < i, 7 < 15} = 1.023 and || y —2© ||= 5.74] 


Lifting Structure and Reversibility 


The notion of reversible transform distinguishes appreciably JPEG 2000 from JPEG 
(which does not know it). Reversibility means exact invertibility in integer arith- 
metic. Thus a reversible transform maps vectors (matrices) with integer coefficients 
onto vectors (matrices) with integer coefficients, and so does its inverse transform. 
Clearly, this is a fundamental device for lossless compression. Unfortunately, this 
notion is a priori hostile to usual matrix computations (i.e. to “geometric” linear 
transformations). Now, the theory which discusses the quality of the transform can- 
didates for (image) compression is wavelet transform theory. So, we will search for 
interesting reversible transforms in the setting of non-linear approximations for the 
wavelet transforms which are optimal — according to their theory. 

The crucial question: How can we find a reversible approximation for a DWT? 

The answer comes from a factorization trick into elementary transformations; it 
is the famous lifting structure.?? 


Lifting Structure 


The theme is to decompose a two channel filter bank into appropriate elementary 
round transforms, thus creating a formal analogy to the DES scheme in cryptogra- 


phy. 


z[n] = the input sequence 
Situation: 

yoln| low-pass 

= the two |. sequences 
high-pass 

yi[n] 

We will get 
yo[n| 


after L rounds of a lifting structure in the following way: 
yi[n] 


23 Famous for insiders — the word lifting comes from a method of design for good 
filter banks. 
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ys” [n] = a[2n] 


yf} fn] = e[2n +1] 


(1): For l odd1<I< L 


L I-1 
ys" [n] = ys [nl] 


yf in] = yf In] +O, Aalilyd @ in - fj 


(2): For l evenl <I<L 


ys? [n] = yf? in] +O, Arley! fn — 


Ll I-1 
yf in] = yf In] 


After L lifting rounds (or “lifting steps”), we obtain 


yo[n] = Ko- yi} [n] 


y(n] = Ki yf} fn] 


where Ko and Kj, are two appropriate multiplicative factors (the gain factors of 
the two subbands). 

Attention: The lifting structure is specified by the L sequences A1[n], A2[n],..., 
Az[n], which will be each, in all interesting cases, reduced to merely two non-zero 
numbers. 


Important Observations (1) A lifting structure is trivially invertible: 


{l-1} {1} 
Yn [n] = Y1—p) [n], 


why In], = vhhIn] — O, Aldlyto ain — a, 
where p(l) = 1 mod 2 is the parity of 1. 

(2) The invertibility of the structure remains unchanged, if the linear operations 
defined by the convolution products with the various A;[n], 1 < 1 < L, are 
replaced by arbitrary operators.?* 


Example Consider the two-round lifting structure given by 


M[-1] =-3, [0] =—3 


24 The situation is here the same as with the cipher DES: two channels, every in- 
dividual round only affects one of the two channels — hence there is invertibility 
without specification of the added “mixing function” f(Ri-1, Ki). 
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(Note that the situation is already typical for most lifting structures of broader 
interest: At the odd step, two non-zero values for n = —1 and n = 0, in the even 
case, two non-zero values for n = 0 and n = 1.) 


Look now at the fina 


l result of the subband transform defined this way: 


ylP¥n] =v Er] = vf fe) — 5 (ahr) + 06 fn +11) 
seal 5 (al2n] | a[2n + 2]) 
=2-( 7en] seen + 1] 7el2n 2}). 
wn] = of fr] + 5 (yl Min] + of" fm - 11) 
Bion ; (e[2n +1] 
5 (o[2n] + o[2n + 2}) + alan — 1] — (xan — 2] 4 n[2nl)) 
$ alm] + 5 (al2n +1] + alan — 1) — 5 (alan +2] + 2[2n— 2). 
We have obtained a factorization of the DWT 5/3 spline into a two-round lifting 


structure 
(with Ko = 1, ki = 3). 


Remark The DWT 9/7 
where 


CDF admits a 4-round lifting structure representation, 


da[—1] = Ai [0] = —1.586134342, 
d2[0] = A2[1] = —0.052980118, 
3[—1] = A3[0] = 0.882911075, 

] 


da[0] = Aa[1] = 0.443 


With K = 1.230174105 we have: Ko = 


Exercises 


(1) Consider the perfect 


506852. 
K 


reconstruction filter bank given by 


ho = (ho[—3], ho[—2], hol], ho [0], ho [1], hol2], ho[3]) 
= 3 3 73 17 73 3 3 
280° 56° 280° 28’ 280° 56? 280 


1 
20° 


1 3 
475) 


1 


hi = (Ai [—2], hi [-1 ,h5 [0], hy [1], 4 [2]) = ( 30): 


Inspired by the “balanced” type of factorization in the two examples above, we 
aim at a lifting structure representation of the following form: 


iL 
a? 


uy. n] = 2[2n] yf [n] = w[2n + 1], 
1 (0) 
me - = ¥9., ni, {0} {0} 
yt [In] =a“ [n] +Aryg 7 [nr] +Aryo “In + 1, 
2 1 1 
ys” En] = yh? fn] + ray}? [in — 1] + ray}? (nl, 
fn] = ye? In 
whe?) = aM uff] = vl? Eo] + Agu) + ol? En + 
yo [rn] = 4% nl + Naya In—1 + ray [rn], gr’ [pe] = yr? (nl, 
y[2n] = Ko- yp [nJ, 
y[2n+ 1) = K yf Yn). 


Show that our program is realizable, while computing \1, A2, A3, Aa and Ko, A1. 
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[Help: Develop first yt in] = ye [n] in function of a “neighbourhood of 
x[2n+1]”, this will give \1, A2, 43 and Ki, by comparison of coefficients. Then, 
determine A4 and Ko while developing ys tn] in function of a “neighbourhood 
of x[2n]”. 
You should find Ai = 0, Ae = z, A3 = aa M4 = aa Ko = 2, ky= a. 
This gives a “false” four-round structure — the first round being trivial.] 

(2) Now let us carry out a truncation of the lifting structure above: Consider only 
the composition of the three first rounds (we shall skip the last round). 
(a) Find the analysis filter bank thus defined (we still keep Ko = 2 and 

Ba, 


| Solution: hg = (ho[—], ho [0], Ao[1]) = (= 


7 
hy = (hy [—2], hy [=1), hy [0], hi (1), h 
Verify that a = 4.8 
(b) We have found a filter bank which admits the DWT 7/5 Burt as an “exten- 
sion”. Compare the effect (of the 2D version) of this filter bank with that 
of the DWT 5/3 spline and of the DWT 7/5 Burt on the two test schemes 


5 


Th Ht 00011000 
09599999 00022000 
32353533 00033000 
Re oe ee ae eer pe re 
55555555 55555555 
66666666 00066000 
CEECTILG 00077000 
88888888 00088000 


Remark Our “amputated” filter bank is not really in the standards of the practi- 
tioner: the low-pass analysis filter is shorter than its partner. If we try to repair 
this deficiency by an exchange of the analysis filter bank with the synthesis filter 
bank, then it is the high-pass filter which will not vanish conveniently on constant 
vectors ... 

At any rate, the correct viewpoint is the following: Consider the elementary 
round transforms as a construction kit for possible filter banks. Then the addition of 
an appropriate fourth round will produce the DWT 7/5 Burt from the rudimentary 
version above. 


[The three variants of the first transformed scheme: 


i Ooch Qh 26. 4 <6 1.29 0 1.29 0 1.29 0 1.29 0 
0000000 0 0.1 0 —0.1 0 —0.1 0 —0.1 0 
Sir 8e Ory tse: O $0.3 0 4 0-3. 0 
0.50 0 Ou. 007-6 7 000000 
%/3- 59505050 5" 5 9 5 05 05 0 
OO OF Oo to .0 0 6: 0.0.0: OOo -0 
7.25 0 7.25 0 7.25 0 7.25 0 Fe i Ae OS FO 
0.5 0 0.5 0 0.5 0 0.5 0 0.7 0 0.7 0 0.7 0 0.7 0 


25 Recall: a is the determinant of the analysis matrix A. 
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1.24 0 1.24 0 1.24 0 1.24 0 
0.1 0 —0.1 0 —0.1 0 —0.1 0 


2.98 0 2.98 0 2.98 0 2.98 0 ; & » 
of 0 8 fh OO 0 In this example, our “reduced 


Y7/5 = e filter bank produces a 
Oe De ae rather honourable result. 
0 0 0 0 0 0 0 0 


7.15 0 7.15 0 7.15 0 7.15 0 
0.7 0 0.7 0 0.7 0 0.7 0 


Now pass to the second test scheme. 


0 0 0.125 0.25 1 —0.25 —0.125 0 

0 oO O 0 0 0 0 O 

0.375 0 0.703 0.656 3 —0.656 0.047 0 

0.75 0 0.656 —0.188 0 0.188 0.844 0 
5 


¥5/3 =~ 475 0 4.781 0.063 —0.063 4.719 0 
—1.25 0 —1.094 0.313 0 —0.313 —1.406 0 
—0.625 0 0.359 1.969 7.25 —1.969 —1.609 0 

0 0 0.063 0.125 0.5 —0.125 —0.63 0 


0 -—0.06 0.18 0.45 1.1 —0.39 0 0O 

—0.2 —0.01 —0.19 0.04 —0.11 —0.03 —0.2 0 

0.57 —0.12 0.92 0.85 2.65 —0.73 0.57 0 

—_ 115 0.06 0.99 —0.4 0.16 0.35 1.15 0 
¥3/5 = 414 —0.04 4.27 0.3 4.88 —0.26 4.14 0 
—1.45 —0.07 —1.24 0.51 —0.21 —0.44 —1.45 0 

O -035 1 245 6 —-21 0 O 

O -—0.04 0.1 0.25 06 —0.21 0 O 


—0.1 —0.07 0.19 0.47 1.07 —0.4 —0.170 

—0.2 —0.01 —0.18 0.04 —0.11 —0.03 —0.21 0 

0.75 —0.11 1.23 0.77 2.69 —0.66 0.63 0 

— 1.16 0.06 0.91 —0.4 0.15 0.35 1.22 0 
Y7/5 = 407 —0.05 4.27 0.32 4.88 —0.28 4.02 0 
—1.47 —0.07 —1.15 0.51 —0.19 —0.44 —1.54 0 

—0.39 —0.387 1.23 2.61 6.16 —2.24 —0.79 0 

—0.01 —0.04 0.15 0.25 0.61 —0.21 —0.05 0 


We note finally: Our two test examples do not give a clear argument for the 
superiority of the DWT 7/5 Burt over its truncated version. 

But there remains altogether the question: Why do we search to obtain a proof 
in favour of the DWT 7/5 Burt relative to the filter bank which is its “prefix”? 
The first answer is simple: If the impulse responses become longer, less localized, 
then the treatment of the digital material should be more refined (which explains, 
by the way, the non-annihilation of certain positions with respect to the action of 
the DWT 5/3 spline). The second answer is of ideological nature: The DWT 7/5 
Burt is a true Discrete Wavelet Transform; this is not the case for its truncated 
version (for a filter bank which realizes a (biorthogonal) DWT you have necessarily: 
So (—1)*ho[k] = 0 — which does not hold here). Now, the principal paradigm of 
our theory is the supremacy of the filter banks which realize a Discrete Wavelet 
Transform over the filter banks which do not. The next section shall be devoted to 
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this subject. We shall see that the arguments of excellence primarily turn around the 
properties of the impulse response gj — and it is not gf which makes the difference 


between the DWT 7/5 Burt and its truncated version. 


(3) Now we eAcTeneS; in (1), the second and the fourth round (i.e. we have A; = 
0,A2 = 100» 43 = ins = 3)" 


(a) Determine the Bite aca: analysis wae h* = (hg, h{). Compute 
do halk], D2(—1)*holk], 2 halk], DO(— 

(b) Finally, ne the analysis filter fois uM the synthesis filter bank. 
Determine the ”dual” impulse responses h* = (ho, hj): You have to write 
x[2n] in function of a neighbourhood” of y[2n] and x[2n + 1] in function 
of a “neighbourhood” of y[2n + 1]. 

Compute )7 ho[k], )7(—1)"holk], Do AI[K], SO(-1)* hi. 

[(a) You should obtain (putting Ko = Ky = 1): 


ho = (Ao[—3], ho[—2], ho[—-1], ho[0], ho[1], Ao[2], ho[3]) 
( 3 1 73 6 73 1 5) 
200’ 14’ 200’ 7’ 200’ 14’ 200 


t _ (pt t t t t = 3 5 17 #5 3 ) 
hi = (hi[-2], Ai-1], Ai(O}, A], MB) = (-,-2,0,-2,-2 
Note the synthesis character of the coefficients: Alternating type of the denominators 
according to alternating membership of g§ or of gt. 
(b) The “dual” case looks much more natural: 
ho = (ho[—3], ho[—2], ho[-1], hol0], ho [1], ho [2], ho[3]) 
_ ( 3 3 73 17 73 3. 3 ) 
200’ 40’ 200’ 20’ 200’ 40’ 200 
hy = (hi[—2], hi[-1], hi [0], hi [1], hr [2]) 


- ( 15 65 1) | 
14°14°77 14’ 14/° 


Reversibility via Lifting Structures 


Recall: The invertibility of the lifting scheme does not depend on the linearity of 
the convolution term in the expression 


fay = yl OD) tn 5 
Yar] = ya [rl + S lily pay ln — dl: 


Hence, the scheme will remain eth when replacing all the convolutional 
terms by non-linear approximations: 


l I-1 l-1 . 
tal = uh nl 5 + Mian che — a 


(where |-| means “integer floor”?”). 

6 The pynuhens ei bank of the DWT 7/5 Burt is given, in lifting structure, by 
Mi = a A2 = qq A3 z, Aa = 0. There should be some similarities. 

27 |a| =max{ne€Z:n< a} 
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By this clever trick — representation as a lifting structure and reversible ap- 
proximation of the round transforms — every Discrete Wavelet Transform admits a 
(non-linear) reversible variant. 


Remark We have decided to use the rounding function r(x) = [5 +2]. 

It differs from the usual rounding function (which is an odd function) on the 
arguments of the form —(n+ 3); n € N: The usual round of « = —3.5 is —4, whereas 
r(—3.5) = —3. 


Exercises 


(1) Let us begin with the reversible version of the DWT 5/3 spline. Show the 
following formulas: 


(a) In analysis: 


y[2n + 1] = 2[2n+4 1] 


(x[2n] + x[2n + 2])| 


Disposing of these values, one computes 


y[2n] = 2[2n] 4 E z(ul2n 1] + y[2n + 1))| 
(b) In synthesis: 


2[2n] = yl2n] ~ [5 + Z(oL2n — 1] + yl2n + 1)] 


Then, as in analysis 
a[2n + 1) = y[2n + 1] + [3 (x[2n] + x[2n + 2])| 


(you have to justify the simplified expression for the floor term in high-pass). 


(2) Now consider the gradation 


30 30 30 30 30 30 30 30 
60 60 60 60 60 60 60 60 
90 90 90 90 90 90 90 90 
120 120 120 120 120 120 120 120 
150 150 150 150 150 150 150 150 
180 180 180 180 180 180 180 180 
210 210 210 210 210 210 210 210 
240 240 240 240 240 240 240 240 


and its transformed scheme — by the DWT 5/3 spline, the DWT 7/5 Burt and 
the DWT 9/7 CDF: 


30 0 30 0 30 0 300 37 0 37 0 37 0 37 0 
00000000 30-3 0 -3 0 -3 0 

90 0 90 0 90 0 90 0 89 0 89 0 89 0 89 O 

2 OEE 08 D0) 0-0 00000000 
¥5/3 ~ 159 0 150 0 150 0 150 0 ¥7/5 ~ 159 0 150 0 150 0 150 0 
00000000 00000000 

218 0 218 0 218 0 218 0 215 0 215 0 215 0 215 0 
150150150150 21 0 21 0 21 0 21 0 
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40 0 40 0 40 0 40 0 
404040 4 0 
92 0 92 0 92 0 92 0 
000000 0 0 


¥9/7 ~ 148 0 148 0 148 0 148 0 
20 H00 =3 03-6 
212 0 212 0 212 0 212 0 
13 0 13 0 13 0 13 0 


(we have rounded the coefficients of 
the three results to integers, 


according to the spirit of the section...) 


(a) Carry out the synthesis transforms on the three schemes above, then round 
the coefficients of the results to integers. Compare with the initial gradation. 
(b) Compute, in each of the three cases, the reversible version. 


[(a) In case of the DWT 9/7 CDF, we recover precisely the initial gradation; in 


case of the other two transforms, we obtain the same deviation on the two last rows: 
210 will change to 211, and 240 will change to 241. 


(b) The three transformed schemes in reversible mode”®: 


30 0 30 0 30 0 30 0 


00000000 
90 0 90 0 90 0 90 0 
gn O 30.008" 00 
¥5/3 = 150 0 150 0 150 0 150 0 
00000000 
218 0 218 0 218 0 218 0 
30 0 30 0 30 0 30 0 


T3073 7350 73 D 
60-60-60 -60 

175 0175 0175 0175 0 
So O00 OOO 0.0 
¥7/5 ~ 994 0 294 0 294 0 294 0 
00000000 

420 0 420 0 420 0 420 0 

42 0 42 0 42 0 42 0 


62 0 62 0 62 0 62 0 
10 1101 101 101 
139 —1 139 —1 139 —1 139 -1 
: 000 00 00 0 
Y9/7 = 994 —1 224 —1 224 —1 224 -1 
5 0 -5 0 -5 0 —5 
321 0 321 0 321 0 321 
26 0 26 0 26 0 2 O 


(3) In the situation of the exercise (2), replace the considered gradation by the 
following “gradation cross on a black background”: 


0 0 0 30 30 0 0 O 
0 0 0 60 60 0 0 O 
0 0 0 90 99 0 0 O 

oe 120 120 120 120 120 120 120 120 
150 150 150 150 150 150 150 150 
0 O O 180180 0 0 O 
0 O O 210210 0 0 O 
0 O O 240240 0 0 O 


28 Caution: We shall put everywhere Ko = Ki = 1 — this explains the surprising 
magnitudes of the coefficients! 
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(a) Compute, for each of our three standard transforms, the integer round ¥ of the 
analysis transform y of x, then the integer round Z of the synthesis transform 
z of y. 

(b) Compute the three reversible versions. 


[(a) First the analysis transforms (with the usual rounds): 


00 4 8 30 -8 -4 0 —3 -2 6 14 32 -12 -5 0 
000 00 0 00 6 0 -5 1 -3 -1 -60 
11 0 21 20 90 -20 1 0 22 -3 37 23 81 —20 19 0 
. _ 23020 -6 0 6 20 , — 35 2 27 -12 5 10 37 0 
¥5/3 > 143 0 143 2 150 —2 1420 7/5 ~ 192 1 128 10 146 —8 1210 
—380-33 9 0 -9 —420 44 —2 —34 15 -—6 —13 —46 0 
—190 11 59 218 —59 —48 0 —14-11 37 78 185 —67 —240 
002 4 15 -4 -20 0 -1 4 7 18 -6 -10 


5 111 9 35 -12 2 3 
303 04 0 3 0 
22 1 34 19 83 —-23 15 7 
.  . 22018 -6 3 7 24 -2 
Y9/7 = 193 0 127 7 145 —8 121 2 
471 —39 12 —9 -15 —51 4 
—5 4 32 58 183 —72 —25 20 
14014 0 13 0 14 0 


Now the rounded schemes of the synthesis transforms: 


0 0 0 31 30-1 0 0 1 0 0 30 29 0 0 0 
0 0 0 61 60 -1 0 0 0 1 1 60 61 0 0 0 
-1 -1-1 91 909 -1 0 0 0 0 0 90 99 0 0 0 
.  _ 121 121 120 120120120119 119, 119 120 119 120 121 120 121 121 
45/3 ~ 151 150 149 151 150 149 151 151. “7/5 ~ 150 150 149 150 149 150 150 150 
-1-1 0 179180 1 1 1 0 1 1 1801790 0 0 
0 0 0 210211 0 0 0 0 0 O 210211 0 -1 -1 
0 0 0 241241-1 0 0 1 0 0 239240 1 0 O 


0 1 1 29 30 -1 1 «#1 
0 0 0 60 6 0 0 0 
1 0 1 91 909 0 -1 0 

_ 120 120 119 120 120 120 120 120 

49/7 ~ 151 149 150 150 150 151 150 150 

0 O O 180179-1 0 0 

0 O O 209210 0 O -1 

0 1 0 240240 0 0 0 


In order to duly appreciate the numerical values of the reversible versions to 
follow, recall that they differ from the numerical values of the standard versions 


: i 4 1: 
(very, very) approximately by a factor K? in subband LL, by a factor Ko-K,; im 


subbands LH and HL, and by a factor = in subband HH. This makes roughly 
1 


(1,2,4) for the DWT 5/3 spline, (2,2,2) for the DWT 7/5 Burt, and (3,2, 5) for 
the DWT 9/7 CDF. 
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(b) The three reversible versions: 


00 4 15 30 -15 -40 -8 —4 11 28 63 —24 —110 
000 00 0 00 13 0 -11 3 —7 —2 -130 
11 0 21 39 90 —40 1 0 42 —7 71 47 157 —40 37 0 
. 45 0 39 -233 0 22 510. 70 3 54 -25 9 21 730 
¥5/3~ 1430 144 3 150 —4 1420 7/5 ~ 938 —3 251 20 287 —17 235 0 
—750—66 37 0 —38 —840 89 —5 —69 31 —11 —27 —930 
~190 11 118 218 —119 —49 0 —27 -22 72 157 362 —134 —460 
00 4 15 30 -15 -40 -1 -2 9 15 36 -13 -30 


6: 1 ae a9 “53° 95> 7 
6 OS 36. Oe OR SO Be 
34 3 51 38 126 —47 23 13 
on -  Ah ea Bee aye “AOS AR. 6 
Y9/7 ~ 187 1 193 13 219 -18 182 4 
03 9 <77 31 =17 —99.—102 10 
-—8 7 50 116 278 —144 —37 40 
97 -1 27 -1 26 1 28 -1 


(4) Consider a “natural” reversible version of the DWT 5/3 spline which is 
obtained as follows: 

First, we note: If T is the matrix of the analysis transform (of arbitrary size) 
of the DWT 5/3 spline, then S = 4T is a matrix with integer coefficients. Hence, 
if X is an integer 2D scheme, then Y = TXT* = qoxs® becomes integer if one 
multiplies all coefficients of Y by 16. This gives the following reversible algorithm: 

Let X be an integer matrix. 

(a) Transform, by the usual DWT 5/3 spline, the scheme X’ = 16X. Y’ = TX'T* 


will be an integer matrix. 
(b) Backwards, transform, in usual synthesis, Y’ into X’, then divide by 16; we 


obtain X = 4X". 
Why will this mathematically correct version not be accepted by the practitioner? 
(5) Do there exist integer 8x 8 schemes which are fixed points for the 2D reversible 
version: 
(a) of the DWT 5/3 spline 
(b) of the DWT 7/5 Burt 
(c) of the DWT 9/7 CDF 
|Help: (b) You should search (and you will find) a fixed point in the integer 
multiples of the matrix 


10101010 
01010101 
10101010 
x _0Otol10l 
10101010 
01010101 
10101010 
01010101 
(a) and (c) Here, the answer is probably “no” — the author counts on the math- 
ematical cleverness of his reader.] 
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5.3.2 The Discrete Wavelet Transform 


Until now, our exposition has given priority to two or three particular filter banks, 
especially the DWT 5/3 spline. Our mathematical curiosity should have become 
excited by the notion of a Discrete Wavelet Transform. 

There seems to be a continuous world — that of the wavelets — which has the 
authority to declare certain filter banks as derived objects. 

Our next objective will thus be to explain why a filter bank which realizes a 
Discrete Wavelet Transform is particularly interesting for the practitioner. 

First, we have to introduce the wavelets. 

At level zero, we have adopted an intuitive and simplifying viewpoint: the sin c 
function is a wavelet — look at its line — and everything similar should be a wavelet. 


At level one, we shall insist on discretization (i.e. on coordinates). The Whittaker- 
Shannon theorem, with variable frequential window, introduces the translates of the 
dilated sin c function (according to the widening of the frequential window) as an 
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orthonormal basis “in packets” of the L?(R), allowing a more and more faithful 
coordinatization (by opening of the frequential horizon). 

And this will also be the general situation: a scaling function, dominating a fil- 
tration structure, accompanied by a mother wavelet, which dominates the associated 
graded structure, will establish the setting for a more and more faithful coordinate 
refinement. We are in the situation of a generalized Shannon theorem: the vary- 
ing frequential window will be implicitly present through the operation of (dyadic) 
dilation. 

Now consider two successive layers of resolution (of approximation): 

For x = «(t) (of finite energy), its approximation 2“""~») at scale 2”~1 and its ap- 
proximation 2‘) at (wider) scale 2” will be connected by 2(°"~) = a 4 d(™—-D 
(interaction of the filtration and of the grading) where the complementary term 
d’"-) describes the “details to the order m — 1”. This equality — which is an al- 
gebraic banality — can be read in two directions: 2°"—~) H+ (a, do-)) and 
(2™, dim) es plm-2) 

If we now consider these operations on the coordinates, we actually obtain a 
two channel filter bank, which does not depend on m. This is the Discrete Wavelet 
Transform associated with a multi-resolution analysis. 

The orthonormal wavelet bases (for example the basis generated by the sin c 
function) are mathematically too perfect in order to be really adapted to finer prac- 
tical needs. So, we have to look for some compromise. Fortunately, when passing to 
biorthogonal wavelet bases (one cleverly splits the approximation scene into analysis 
filtration and into synthesis filtration) one resolves all problems which the practi- 
tioner could pose for the theoretician. 

At this point, we no longer can avoid the question: Why are we interested making 
sure that a given filter bank is actually a Discrete Wavelet Transform? 

The answer is the following: 

It is the degree of regularity of the (synthesis) mother wavelet which controls 
the quality of the DWT associated with the considered biorthogonal wavelet bases. 
On the one hand, in analysis, we obtain the expected trivialization of very regular 
numerical data (they will produce no details). On the other hand, in synthesis, we 
obtain a neatly localized propagation of local errors. 

A filter bank which is a DWT thus allows a qualitative evaluation a priori. Our 
presentation will remain essentially temporal (or rather spatial). Only for the design 
criteria towards “good” Discrete Wavelet Transforms we are obliged to adopt a 
spectral viewpoint (which is, at any rate, more technical than conceptual). 


Recall Our temporal (spatial) world will always be real. 


Multi-Resolution Analysis and Wavelet Bases 


Our formal setting will be the Hilbert space L?(IR) of (classes of) square integrable 
functions on R. 


Recall: The Whittaker-Shannon Theorem 


Recall the principal result of our third chapter (on sampling and reconstruction of 
signals): 
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A square integrable bandlimited (hence essentially smooth) function is deter- 
mined by an appropriate sequence of its values (i.e. by an equidistant sampling 
which is “sufficiently fine”). More precisely: 

Let T > 0, and let hr(t) = sine(4) [= ae with continuous prolongation 
at OJ. 

Visualize: The smaller is T, the more the renormalized sin c function is “ner- 
vous”; the larger is T’, the more it is “flattened”. 

Let x = x(t) be a function which is square integrable on R. Assume that @(w) = 0 
outside [— 4, 7]. Then x = x(t) is essentially smooth (i.e. equivalent to an infinitely 
differentiable 11 = x1(t)). Let us suppose that « = x(t) is regular from the beginning, 
then we have: 


x(t) = So a(k-T)hr(t-k-T) 


[Note that here we have a convergence in quadratic mean. But the Shannon interpo- 
lator is the (inverse) Fourier transform of a function with bounded support, hence 
infinitely differentiable. Our hypothesis allows an identification as functions.] 

We insist a little bit on the ingredients of the interpolation formula: if 


a(t) = FF[x(¢)], 


then a(k-T) = 4(x(t),hr(t-—k-T)) = 4(a* hr)(k-T) [Concerning the notation: 
t is here an integration variable, hence logically bound (“annihilated” by (,)).] 


Remark For fixed T > 0, let Wr = {x € L?(R) : &(w) = 0 outside [-4, Z| } , 


The orthogonal projection of  € L?(R) on Vr is 


rp = Fuxhr 


The argument: 
Let xr € Vr be the nearest element to x. Then we have, by the Plancherel 
formula: 


1 r a 
Ic —e7||? = x/ | #(w) — &@r(w) |? dw 
1 


=F | (w) |? dw + — | #(w) — @r(w) |? dw. 
|wl>n/T Jwl|<a/T 


This distance is minimum in case @r(w) = %(w)I|_z,2)(w) = 4a (w)hr(w). [Note 
that the orthogonal projection will carry out a smoothing: xr is infinitely differen- 
tiable (since hr is).] 

We make T > 0 vary appropriately, and obtain a filtration of L? (R): 

Put V™ = {x € L?(R) : #(w) = 0 for | w |> sat}. We get a chain of closed 
subspaces of L?(R): 


EVE CV) CVO! CVD CVO Coe 
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For arbitrary « € L?(R), m € Z, let x € V™ be the orthogonal projection of « 
onto V‘™). According to the preceding remark, we have ||x“") — «||? 


= Jigen | &(w) |? dw ice. limm——oo |la°™ — || = 0 . In other words 


Unez VO =L*(R) (1) 


On the other hand, we have ||a° ||? = | &(w) |? dw < 2~-™ - const. 


1 
on Sissi 
Hence limm-—c ||v || = 0 . This means that 


Amex V°" =0 (2) 


The filtration is exhaustive (upward and downward). Now, let us look more 
closely at scaling operations. Since F[a(2~™t)] = 2”4(2”w), we have: 


xz =2(t)e VO me=a2(2-™t)€eV™ (3) 


Since F[x(t — k)] = e7'**(w), we have: 


tr = 2(t—k) € VO 


_ (0) 
c=x2(thEeVv Be Z 


Finally, consider the function?® y(t) = sin c(t). We have already seen: 


Let yr(t) = y(t — k), then (yx (t))xez (5) 
is an orthonormal basis of V . 


Consequence (of (3) and (5)) Put yt (t) = V2-™y(2-™t — k). Then 
(po (t)) kez is an orthonormal basis of V“" (in the usual Hilbertian sense). More 
explicitly (we deal with the Wittaker-Shannon theorem applied to the orthogonal 
projection onto V(™): for « = a(t) € L?(R) we have: 


EOS as oO: 
with 
ys” [k] = (a(t), po (t)) = (a * pf”) (2k). 


Remark How can we interpret this “scaled” version of the Whittaker-Shannon 
theorem? 


2° The change of notation h(t) + y(t) indicates an important change of viewpoint: 
an “impulse response” promotes to a “scaling function” . 
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Accept, as a starting point, the following paradigm. 

In spectral analysis of a signal (i.e. in its frequential representation) “the crude 
information is in the low frequencies”, whereas “the fine information — the informa- 
tion on the details — is in the high frequencies” (for the intuitive: think of the left 
hand and of the right hand of a classical pianist). 

We insist: The higher the frequency, the finer are the supported details. If we 
adopt this information hierarchy, we can perceive the approximation of x = z(t) 
by 2) = o(™ (t) as a “level m resolution”, which substitutes for the initial signal 
while neglecting the “finer details”. 

Let us be more precise. 

On the frequential side, our resolution horizon is given, in a very suggestive 
manner, by a “truncation window”: it is a real horizon. 

On the temporal side, the resolution screen acts via “digital density”: it is char- 
acterized by an equidistant sampling adapted to the demanded precision of details. 

Hence, the higher (finer) is the resolution, the larger is the frequential window 
(the horizon), the denser is the sampling (i.e. the smaller is the sampling step). 

Some conventions of language: In the version of the Whittaker-Shannon theorem, 
as cited above, the value T for the sampling step will be called the scale, and the 
reciprocal value z the resolution. 


Small scale = high resolution 


Hence: : 
Large scale = low resolution 


In our enumeration of the resolution levels, we have associated with the resolu- 
tion level m the values 
2” = the scaling factor, 
2~™ = the resolution (value). 
Note finally the fundamental role of the function y(t) = sin c(t) as a coordinate 
generator, by means of its translated-dilated variants, as indicated by the Whittaker— 
Shannon theorem. 


Now we can generalize. 
Multi-Resolution Analyses of L?(R) 


Definition (S. Mallat) A multi-resolution analysis of L?(R) is given by a filtra- 
tion (of a chain of closed subspaces) of L?(R) 


HH CV@™ CVM eVOeVDMevo ec... 


with the following properties: 
(MR-1) Umez VW’ = L?(R) 


(MR-2) Alege w?=0 


(MR-3) g=a2(t)€EVO => pe=a2(2-™t)ev™ 
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(MR-4) z= a(t) € VO rp, = a(t—k)€e VO fork eZ 


There exists (one declares) y = y(t) = the scaling function 
(MR-5) (of the multi-resolution structure), such that, with yz (t) = 
y(t — k) we get in (yx (t))pez an orthonormal basis of V . 


Consequence (of (MR-3) and of (MR-5)) Put yt (t) = J2-™y(2-™t—k). 
Then (oi (t))kez is an orthonormal basis of V“. 


Remarks Consider, for « = x(t) € L?(R), its orthogonal projection #°™ = a) (t) 
onto VO, 2(™ will be called the approximation 


om Aa f x. 
of resolution2-™ © ” 


{ at scale2”’, 

v(™ is then the vector space of the “approximations at scale 2”. 

We note the essentially discrete character of the notion of multi-resolution analy- 
sis for L?(R): The pivot of the structure is clearly the scaling function. It is the scal- 
ing function which supports the transition from the continuous world to the digital 
world: the introduction of coordinate systems allows a more and more faithful digi- 
tal representation (refine the resolution) of the approximated signal. In other words, 
the structure of multi-resolution analysis defines the appropriate formal setting for a 
variable digitization via dichotomic refinement of the resolution grid. From another 
viewpoint, we are simply in face of a generalized Shannon theorem. 

The dilations by the factors 2”, m € Z, formalize implicitly bandwidth variation, 
whereas the translations of step 2”, m € Z, formalize equidistant sampling grids. 
Note that this translation — dilation twin reflects perfectly the time — frequency 
coupling which promotes wavelets theory so gloriously. 

Some commentaries concerning the axioms stated above: 

(MR-1) means that every « = 2(t) € L?(R) is determined by (the knowledge of) 
its (high) resolutions: x = limm—-—co gh™ . 

(MR-2) allows a conceptual interpretation that we shall see in a moment... 
(MR-3) formalizes — in quantized form — the frequential aspect of the transitions 
between the resolution levels. 

(MR-4) formalizes — in quantized form — the “equidistant grid” aspect inside the 
particular resolution levels. 


(MR-5) is the digitization axiom, fundamental for the notion of multi-resolution 
analysis. It indicates, in which way the Vvi™ are effectively coordinate spaces. 

Let us point out that the notion of orthonormal basis (for the introduction of 
coordinates) quickly reveals itself to be too restrictive for really interesting applica- 
tions. We shall soon find the remedy. 


Exercises 


Piecewise Constant Approximations. Consider 
V™ = {y = y(t) € L?(R) : y(t) = y(k- 2™) For k-2™ <t < (kK4+1)-2™,k EZ}. 
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v'™ is the vector space of square integrable functions which are piecewise con- 
stant on the intervals — positioned in “natural dichotomy” — of length 2”. 
It is obvious that --- CV? CV CVO CVEYD CVO?) C... 


(1) First the formal trivialities: 
(a) Verify (MR-3) and (MR-4). 
(b) Verify (MR-5), with the scaling function y(t) = Io1;(¢) (the unit “time- 
window”). 
1 for—-1<t<1l 
(2) Consider the function x(t) = { 1 = =o 


= else 
lt] 


a) Compute ||z||, the norm of « = x(t) € L?(R). 

(Leto O= oye we « T.om ce41)-am(E). 
Find the values ee, k,m eZ. 

c) Show that limm——o«”(t) = x(t) and that limm—sox'™(t) = 0 (for 
what type of convergence?). 

(3) Let us return to the general situation of a piecewise constant approximation. 

a) Show (MR-1). 

(b) Show (MR-2). 


(Help: For real-valued a = x(t) € L?(R) we have: 


1 (k+1)-2™ 
eG): = Se a / w(t) dt © Ippo cepay.amy(t): 
k-2™m 


keZ 


We obtain: 


+oo 
Iz — &™ |]? = fal]? - |x |? = ‘| n(t)?dt —2"- S°(ue)?, 


keZ 


with w™) = P&I" oaydt. 


So, we need: 

(ay t = Tag 0 a) 

This is a bifurcation of mathematical rigour: Either you consider this approxi- 
mation of the integral as obvious (and a formal banality), or you have to 
prove it.] 


Wavelet Bases 


From the conceptual viewpoint, the notion of multi-resolution analysis (and that of 
scaling function) are fundamental for the organization of the arguments around the 
hierarchical treatment of digital information. 

But from the viewpoint of mathematical formalization, it is a derived struc- 
ture which is more adapted to guarantee algorithmic solutions: that’s the associated 
graded structure, the concrete appearance of which is realized via wavelet bases. 

So, let us begin with a multi-resolution analysis 


CV! CV) CVO! CVD CVO Coe 


and consider the spaces wi”) = vim 2) py) | m € Z. Conceptually, here we have 
introduced the spaces of the “details at scale 2"~'”. We shall realize them as the 
orthogonal complements of the V“™ inside the V°"—»): 
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VeDoVMew™, meZ. 


Let us iterate the decomposition: 


VrD — yt?) go w™ owt) a... Wwlnt?) p>0,meZ. 
At this point the reason for the axiom (MR-2) becomes apparent. Pass to the limit, 
with p> co: VT“) = Ba WwW") ie. every V'"~» is the orthogonal direct sum 


of the W®),» > m (in the Hilbertian sense: as a topological closure.). Finally, with 
m — —oo, we obtain L?(R) = Diez Ww), 


p2=m 


We note: The situation of a multi-resolution analysis now appears in a linearized 
version: the resolution refinement (let us look closer and closer) is replaced by a 
procedure of accumulation of details (forget only what’s not needed). 


Example The Shannon multi-resolution analysis. 


Recall: V'™ = {x € L?(R) : @(w) = 0 for | w [> 277} 
Hence, W'™ = {x € L?(R) : &(w) = 0 outside the band 27" <| w |< 
2-™+171 (this is immediate, by Parseval’s identity). 


Exercise 
Put op” (t) = V2-™ - cos (30 : 27h) sinc (5 : Ort) Then: 


Rake cy J 2 Tor 2 ae ee 
Ce oes . else. 
(b) [Jo |] = 1. 

Qnt at 


: : : : sin( 27) sin(St) __ Banta Tt 
First verify the identity aT aT cos sinc ( ) 


Now consider the translates wer (t) = y™ (t—k-2™) k € Z. Then we have, for 
a = x(t) € L?(R) 


(A) oY) —o™ = SY “(a(t), ver Hk” 
keZ 
The argument: 


(i) First, ( ey ed is an orthonormal system in W‘™: 
pen) (w) = Pee Ge (w) keZ 


(and Parseval’s formula yields, once more, the claim). 
(ii) Then, let us establish a development 


alm—1) = alm) = So eb”. 


keZ 
Write 


alm—1) _ Xe anger?) 
keZ 
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alm) = = So beGe a(m) 


keZ 


a(m —ik2™-1y (m1 
pt ) = J2- e ne 7 gt ) . I 27M a 2M] 


(m—-1) a(m) : 


ie. 2°” is a “restriction” of & as well as ~; ’ is a “modulated restriction” 


of ge )) This gives: 


g™ = So an Ger Tama a-ma] = an s5 Segre res, gm = = So Pee 30) 


keZ keZ keZ 


We have thus: a, = V2: eT ik2™ hw - Be k € Z. Finally, we obtain what we 

m— a(m —ik2™-ly a(m— a(m 
want: ¢! G ah Vi Seu he Ke -Be-@ : 1) = Vrez BrP (m) _ 
Fae Bude” ; 


(iii) The identification of the “coordinate” is now a formality: 


Be = (2-9 — 8 Be) = (B— 8, BI) = BBL) — 2, 


(since #™) is in the orthogonal complement of Ww), So, we have established 
the identity 


HD — a) = SMa), OE)”. 


keZ 


Let us sum up. For every k € Z, the system (bo nen is an orthonormal basis of the 
vector space W‘™). But we know already: L?(R) = Diez W”). Hence, (be Vie men 


is an orthonormal basis of L?(R). 
We insist: Every 2 = x(t) € L?(R) admits a unique approximation (a unique 
development) 


c= S~ (a(t), oh” (yy. 
k,meZ 
Supplementary observation, which is rather important: 
Put u(t) = cos 2rt- sinc (5t) 
Bea Be Viorel lah 
| ue) = 0 else 


Then Pret t) = V2-™ .W(2-™t — k), k,m € Z. In this sense, our orthonormal 
basis is generated by a single function w(t), with the help of a “dyadic quantization” 
by dilation (frequential aspect) and translation (temporal aspect). 


Our particular situation deserves generalization: 


5.3 Filter Banks and Discrete Wavelet Transform 381 


Definition A wavelet basis of L?(R) is given by: 
(1) A mother wavelet w(t), 
(2) A family of translates — dilates: ym (t) =V2-™ -Y(2-"t—k), k,m eZ, 
which is a basis of L?(R): Every « = x(t) € L?(R) admits a unique development 
25 5) Re” 
k,méZ 


Caution A wavelet basis is not necessarily orthonormal! 
Wavelet Bases Generated by a Scaling Function y(t) 


Let us fix our initial data: We start with a multi-resolution analysis 


LHe? CVO EMO EN ON CVO Cin: 


and its scaling function ¢y(t). 

Our objective: Find a mother wavelet w(t) and the wavelet basis (be Ji mez 
which is derived from it such that, for every m € Z, the sequence of the trans- 
lates (be ner is a basis (in the Hilbertian sense) of W'™, i.e. of the orthogonal 
complement of V™ in VO"—), 

Strategy: Find, from the coordinates of y(t) in veo , “appropriate orthogonal 
coordinates” which will define w(t). 

In detail: 

( 


(1) VO cv, hence y = y(t) admits a development with respect to (OO? Jeez: 


g(t) = V2-S © golk]p(2t — k) 
keZ 
(2) Characteristics of go = (go[k])xez 
— under the hypothesis of the orthonormality of (y(t — k))kez : 
+co A S 4 : 
(oo, ox) =2 [7 (5., golile(2e— 4) 32, gol] o(2@— 8) — #))de= 
ci . +oo : - 
257 ,,; glélgolj — 2k] [7 p(2t — i)p(2t — j)dt = 
2>,, golilgoly — 2k] - $(p'-Y,~) =, golilgoli- 2k] EZ. 
We observe: go = (go[k])nez is a sequence in 1°(Z), of norm (of energy) equal 
to 1, which is orthogonal to all its 2-translates. 
And what about the odd (integer shift) translates? Put gi = (gi[k])xez with 
glk] = (-1)**'go[-k +1], k € Z. Then g; and its 2-translates fulfill their 
mission. 


Exercise (Important) 


Show that the 2-translates of go = (go[k])xez and the 2-translates of gi = (g1[k])xez 
constitute together an orthonormal basis of 1?(Z). 

(The orthonormality of the considered system is easy; what we have to show is 
that every sequence in 1?(Z) admits a development in the 2-translates of go and of 
81). 

[ etn: Let then « = (x[k])xez € 1?(Z). We have to show that there exists 


(a[n])nez, (b[n])nez, such that 2[k] = S>) a[n]go[k — 2n] + D>, b[n|gi[k — 2n], 
kKeEZ. 
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Recall (Two channel filter banks — exercises) A sufficient condition for an analysis 
filter bank (h§, h{) (interleaved formalism) to admit a synthesis filter bank (9, gi) for 
a perfect reconstruction is the validity of the identity (4) }*, h6[k](—1)*hi [2n—k] = 
5n,o- In this case, gj and gj will be the “alternating versions” of h{ and of hj. Passing 
now to the conventional formalism — h{[k] = hifk — 1] and gi[k] = gi[k + 1] — we 
obtain the identity (4) 5>,(—1)*** 91 [k]go[2n + 1 — k] = dno. 

Accept the extension of our results from 12(Z) (finite support) to ?(Z) (or prove 
it: it is precisely our strong hypothesis det A = 1 which facilitates the argument 
...). Then our couple (go,gi) is a synthesis filter bank which admits a perfect 


reconstruction, and we can conclude ... 


(3) Put} v(t) = V2- Yonex silk] y(2t — k) 


and, as usual, wl (t) = V2-™.y(2-"t —k) k,m © Z. Let us show first that 
(we (t))nez = ( () (t)) pez is an orthonormal basis of W). 


[Exercise: Show that 


(a) vit) = Oo, golk — Je) 
(b) 25 (t) =o, alk — 27 ¢ PH) | 


(a) Orthonormality 
(Wem) = D>,,, li — 2k] l7 — 2m] (—\-, eS) = I, ali — 2k] gu fi — 2m] = Se,m. 
(3) Orthogonality to V© 


(sm) = Yo,,; 91[ — 2k] gol7 — 2m], P—) = YO, gulé — A] golé — 2m] = 0. 


Hence: (ax (t))kez is an orthonormal family in W), the orthogonal complement 
of VO in VOD. 


(y) Density 
V‘—» is generated (in the Hilbertian sense) by (yx(t))kez and by (wWe(t))kez- Con- 
sider « = a(t) € VO: & = ae ys? [keo?. But, (ys? [k])xez € 1?(Z), and the 


2-translates of go as well as the 2-translates of gi constitute together an orthonormal 
basis of 1?(Z). We thus can write: 


s= (= y [dlgolk — 24) + Sy Gian tk - a) ie 
k a j 
= Dw de? + > Ta les” 


All that is naturally inherited by the dilated versions of w(t), which finally gives the 
claim: 
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( OP) (8), m€Z is an (orthonormal) wavelet basis of L?(R), 


generated from the mother wavelet (t) = /2- onez 111k] p(2t — k) 
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where gi = (gi1[k])xez are coordinates orthogonal to the coordinates go = (go[k]) kez 
of the scaling function y(t) in V~). We insist: 


Let y(t) be the scaling function of a multi-resolution analysis, and 
let w(t) be the mother wavelet for the associated wavelet basis. 


Then the coordinates go = (go[k])xez of y(t) and the coordinates 
gi = (gilk])xez of u(t) in V~? constitute the two synthesis 


filters (go, g1) of a perfect reconstruction filter bank. 


Exercises 


(1) 


(a) 
(b) 


Show — with the notations from above — that 


(eo ,eD) = go[n — 2m] 


WO, pS?) = giln— 2m], mnEZ 


(Caution:We suppose the system (yx)xez to be orthonormal!) 


(2) 


Consider the Shannon approximation. 
Find go = (go[k])xez and gi = (g1[k])xez in this case. 


Result: 

900] = J 

go|2m] = 0 m0 
go[2m + 1] = 2. (-1)" 545 


Now let us pass to the piecewise constant approximation. 


Find go = (go[k])xez and gi = (g1[k])xez in this case. 
Produce the wavelet basis (bo (t)) mez: 


[ Result: 
V2 
= m=0,1 
Jo [m] = {e else 
- m=0 
alm]= 4 2  m=1 
0 else 
aVSrm EO" ht Sho 4a 
mr = 2 Jom EOL <tc (k+1)-2" 
0 else 
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The Discrete Wavelet Transform 
The Orthonormal Situation 


That’s the situation that we have exclusively considered until now: The discretiza- 
tions (the coordinates) are given by development with respect to orthonormal bases 
of the considered approximation spaces. 

Let us underline the principal reason — let alone our “geometrizing instincts” — 
for this mathematical modeling which insists upon orthonormality: Since the norms 
of the functions (in L?(R)) are equal to the norms of the series of their coordinates (in 
1?(Z)), we get isometric coordinatizations. This frequently is called the “numerical 
stability of the discretizations”. That’s a theme which we will not abandon in the 
sequel. 

Let us sum up: 


(1) Every multi-resolution analysis gives rise to a wavelet basis. 

(2) The construction of the wavelet basis generates simultaneously a two channel 
filter bank. 

(3) The principal ingredients of the construction: 


(a) Let y = y(t) be the scaling function of the considered multi-resolution analysis. 
Find its coordinates in V~): 


e= >  golkley” 


keZ 


(8) go = (gol[k])xez will be interpreted as the low-pass synthesis filter of a two 
channel filter bank. gi = (gi[k])xez is derived as the “alternate” sequence, 
orthogonal to go, and defines 

(y) & = W(t), the mother wavelet of the associated wavelet basis: 


b= So alkle, ? 


keZ 


Remark The sequences go = (go[k])xez and gi = (gi[k])xez are, a priori, in ?(Z). 
We shall tacitly suppose them finite (without always underlining it), by practical in- 
stinct and a weakness for simple reference formularies. Altogether, there are themes 
intrinsic to the mathematical theory (for example finite bandwidth in Shannon ap- 
proximation) which are structurally hostile to finiteness of impulse responses. 


Lemma In the situation described above, we have: 


(a) p(t) =, golk — 2nJol™-? (t) 
(b) bY (t) =>, glk — Anjo") 


This is a (trivial) generalization of a previous exercise. 


Important Observation The systems of coordinates of (ower and of 
(bo) nex in V’"— do not depend on m. 

This is clearly a consequence of the generic character of the scaling function y = 
y(t) and of the mother wavelet ~ = w(t) for the coordinatization of the situation. 


We thus obtain — on the coordinate level — a two channel filter bank which provides 
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the transit between consecutive resolution levels — and this independently of their 
location in the dyadic multi-scale hierarchy. 
More precisely: In a coordinate-free setting, the situation is described this way: 
The (orthogonal) decomposition V°"-) =V'™ gw™, 


[More generally: V"—) = VO"*?) g W™ GB WO™ED @... BWI?) 


reads as follows: For x € L?(R), the approximation 2°"~) at scale ae ' de- 
composes into the (rougher) approximation x™ at scale 2” and into w'” , the 
detail-component at scale 27". 


(m1) at scale 2”! decomposes into 


(m+1) 


[More generally: The approximation x 
the (rougher) approximation x(t?) at scale 2+? and into (w'™, w 
wimtP-3)) the detail-component at scales 2”~!,2~*,...,2™+?-1] 


ney 


All this is, at a first glance, a formal banality. 
But let us pass to the coordinates; we shall compute the coordinates of x 
in function of the coordinates of x” and of w'”): 


First c6™—); g(™-) — ye ym Dike” ) Then, 2°”) and w\™: 


2 yl no”, 


ae 3 a 


(m—1) 


Hence: 


eae = Si les Dea isles 
= SW ©) fgatk— 2n] +9 flare — ano". 


Comparison of the coefficients: 
or? Tk] = Sof” frlgol& — 2n] + S° yt” [nJoulk — 2n], EZ 


In compact form??: 


yr) = (t Qu”) x go + (Tt 2yl”) *g1 | meZ 


In other words: The reconstruction of the coordinates of an approximation at 
scale 2”~' from the coordinates of the (rougher) approximation at scale 2” and of 
the detail-coordinates at scale 27! is carried out by means of the synthesis filters 


3° Recall: Down-sampling | 2 and up-sampling ¢ 2 to the factor 2 are defined this 


way: (1 2f)(n) = fn) and (7 2f)(n) = a fos a 
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(go, 1) of a two channel filter bank. The important feature is that the intervention 
of the filter bank is uniform (i.e. independent of the transit level m — m — 1). 

The analysis filter bank (ho, hi) which produces, on the coordinate level, the 
decomposition 2°"~) 1 (a, w™) (still uniformly in m), is given by the recon- 
struction formulas (discussed previously in the exercises on finite impulse response 
filter banks): 


ho[n] = (-1)" ln + 1] = go[-n] 
hi[n] = (-1)"**go[n + 1] = gi[-n] 


(Caution: We have adopted the conventional, non-interleaved notation) 
So, we have, with the notation introduced above: 


yf =| 2(yl"-? x ho) 


yl =| yh"? * hi) 


Definition The Discrete Wavelet Transform associated with a multi-resolution 
analysis of L?(R) is the perfect reconstruction two channel filter bank with synthesis 
filters (go, g@1) defined by the coordinates (in V‘—) ) of the scaling function y = y(t) 
and of the mother wavelet w = w(t). 

A first consequence of our definition is an invasion by the language of (digital) 
filters. Lowering the resolution level (increasing the scale) corresponds then to a low- 
pass filtering: The low-pass subband receives the lower resolution, and the high-pass 
subband receives the details for the reconstruction of the finer resolution. 

Iterating the considered filterings makes the approximations “thinner and thin- 
ner”, while “thickening” — by additive accumulation — the reconstruction details. 

Look at the corresponding notation: 


lB) = gmt) 4S Syl pylon 


OSi<p neZ 
. m m+ m+ 
with 2+?) = yoaee ys PY jpn?) 
Note that, with Ho(x) =| 2(x * ho), we have yor t?) as ba Qe): 


Remark The most appreciable property of the Discrete Wavelet Transform is its 
uniformity: it does not depend on m, the resolution level of the considered approx- 
imation structure. This causes its simple iterative intervention. 

On the other hand, we should refrain from calling it recursive; only its iterative 
application has the recursive aspect of every iteration (like the computation of the 
nth derivative of a function, for example). 


Exercises 
(1) Consider ge = (90[0], go[1], go[2], 9o[3])** with go[0] = —+, go[1] = $, go[2] = 2, 
go[3] = 10° 


31 We only note the values of the support. 
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(a) Determine gi, ho and h, according to our construction rules in the orthonormal 
case, and verify that we thus obtain a perfect reconstruction filter bank. 

(b) Find a non-trivial permutation of the four values above which gives another 
perfect reconstruction filter bank. 


(2) Carry out, in the situation of exercise (1), an up-sampling to the order 3: 


We consider 
gS? = (gf [0], 96? 1, 95” [2], 96? [3], 96? [4], 95” [5], 9$ [6], 96? [7], 95” [8], 98” [9]) 


with 9)” [0] = —35, 99 [1] = 95" 2] = 0, 95°13] = $, 90 [4] = 95°15] = 0, 
= 3, 9) 17] = 98 [8] =, 9§ [9] = &. 


(a) Verify that we obtain — with g®?, h®) and nh) derived from 2) — still a perfect 
reconstruction filter bank. 

(b) Let us admit, in the situation of exercise (1), the existence of a multi-resolution 
analysis which allows us to recover the considered filter bank as the associated 
DWT. Let y = y(t) be the corresponding scaling function, according to the 
axiom (MR-5). 

Put y(t) = 4(4). 
Is y = y(t) the scaling function of a multi-resolution analysis such that our 
“up-sampled” filter bank is the associated DWT? 


(3) Let now go = (g0[0], 9o[1], go[2], go[3]) 


be given by go[0] = 252, go[1] = 2, go[2] = 422, go[3] = 4. 


(a) Verify that 5°, go[k] =1 and that S>, (—1)*go[k] = 0. 

(b) Determine gi, ho and h; according to the imposed identities in the orthonormal 
case, and verify that we thus obtain a perfect reconstruction filter bank. 

(c) Consider go(w) = go[0] + go[1Je*” + go[2]e~%” + go[3Je~3”. 
Accept the following result: If | go(w) |> 0 on [—§, 5], then go is the low-pass 
synthesis filter of a DWT associated with a multi-resolution analysis of L?(R). 
Verify the validity of our go. 


The Biorthogonal Situation 


We have just seen: In the orthogonal situation ((Yn(t))nez is an orthonormal basis 
of Vv), the analysis filters and the synthesis filters of the considered DWT are 
trivially connected: 
(holn], ha[n]) = (gol[—n], g1[—n}) 

On the other hand, the two “star” wavelet transforms that we have encountered 
within the JPEG 2000 setting, the DWT 5/3 spline and the DWT 9/7 CDF, do not 
have this property (they are not orthogonal). 

Actually, they present (for image processing) practically indispensable general 
characteristics, which reveal themselves to be hostile to orthogonality. 

Let us explain this. 
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First, there is the finite support. It is unimaginable not to use finite length im- 
pulse responses in practical applications (except if they are theoretically indefensible; 
in this case, we altogether will have to live with appropriate truncations). Now, finite 
support alone is perfectly compatible with the orthogonality of a filter bank. 

But add the demand of symmetric impulse responses (this means linear phase) 
which is a must by the very philosophy of image compression: the transition to a 
ower resolution should numerically be done by computation of local means, i.e. with 
the help of symmetric barycentric formulas. Unfortunately, we have the following 
result: 


A two channel orthogonal filter bank (in the sense of the preceding section) 
which has impulse responses of finite length and of linear phase (i.e. they are 
symmetric or anti-symmetric) cannot admit more than two non-zero coefficients 
per impulse response 


Hence we are obliged — if we want to work both with finite length and 
symmetry — to consider filter banks of a more general type. 
But let us begin first with a 


Remark We shall be obliged to give up the geometric paradise of orthogonality for 
practical reasons; but we shall try to counterbalance this loss by a maximum of 
formal comfort: 

On the one hand, we shall try to stay as near as possible to a geometrical 
language (faithful to inner products “with a meaning” ) — this also will be dictated by 
practical necessities (numerical stability of the coordinatization); on the other hand, 
we shall strongly insist upon our principal hypothesis: the coupling of finiteness and 
symmetry for the considered impulse responses. 

Hence: The results of the section on the (finite impulse response) filter banks 
will always be the basic formulary for our arguments. 


Recall Let (hj, h{) be the couple which defines the analysis filter bank, and let 
(gi, gi) be the couple which defines the synthesis filter bank’. 
Then it suffices to specify hi, and gf, satisfying?? 


a forn=0 
Si ritelbion m= {5 Se 
k 


hj and g} are then computed by the formulas 


If we search for a generalization of the orthogonal multi-resolution analysis concept, 
while refusing to give up the familiar formal setting — and aiming at a naturally 
associated DWT — we have to proceed as follows: 


32 ()' = interleaved notation. 
33 q, the determinant of the analysis matrix A, will be, in general, normalized: 


_ 1 
a= 5,lor2. 
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First, we need an alleged version of the axiom (MR-5): 
There exists (one chooses) y = y(t) = the scaling function 
(MR-5)* (of the multi-resolution structure), such that, with y(t) = y(t — k), 
we have in (yx(t))kez a Riesz basis of VO . 


The notion of a Riesz basis is a natural generalization of the notion of an ortho- 
normal basis (in the Hilbertian sense): from the numerical viewpoint, the big advan- 
tage of the orthonormal bases comes from the inner product coordinates; hence we 
shall impose a robust condition which will maintain the continuity of the coordinate 
functions and which will allow the intervention of the Riesz theorem (reflexivity of 
Hilbert spaces). More explicitly, in our situation: 


(i) Every x = a(t) € V admits a unique development « = Sen yo [k]pr 
(ii) There exists 0 < A < B, only depending on (Yx)xez, with the estimation 
0 0) 2 + 0 
A-[lys |? < [le l? < Be luo I? ie. Mle? < XL ye?) PS a lel? 
for alla eV. 


This definition deserves a commentary: Visibly, we have here an equivalence 
of norms (on the space L?(R) and on the space 1°(Z)). We thus obtain a homeo- 
morphism between V) and I? (Z) — which yields the “numerical stability” of the 
coordinates. Note also a finer aspect of the definition: The family (yx)xez is an un- 
conditional basis of V): for all sequences (s%)zez in 1?(Z), the sum do, sep(t — k) 
exists in V and does not depend on the summation mode. 

Note finally that our readjustment of the definition of a multi-resolution analysis 
by the relaxed condition (MR-5)* is amply sufficient for all interesting applications. 


Attention The Riesz basis postulate for vo puts immediately its dual basis 
(represented by vectors of the vector space Vv) at our disposal. But it is not at 
this basis that we aim with the notion of biorthogonality ... 

Now let us come back to our problem: How can we generalize the orthonormal 
situation in order to get the formal freedom for a Discrete Wavelet Transform which 
is described by a filter bank with finite and symmetric impulse responses? 

Let us begin with splitting the multi-resolution analysis of L?(R) into two copies: 


(VO) nex = the synthesis approximation, 


(VO) nex = the analysis approximation. 


Let » = y(t) and ¢ = G(t) be the two scaling functions for the coordinatization 
in both cases. 


Remark It is sufficient to specify V© and V© with their Riesz bases generated by 
y = y(t) and ¢ = g(t). All the remainder will follow: 


r=x(t)ev™ or = «(2™t) € VO 


z=ax(t)ev™ ot = 2(2™t) € VO 
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Concerning the synthesis filter bank (go, gi) and the analysis filter bank (ho, hi) 
of the associated DWT, the impulse responses will be the coordinates (in v(-) and 
vO) of the considered scaling functions and of the mother wavelets: 


p(t) = V2- >>, golk]y(2t — k) 
v(t) = V2- 39, m[k]y(2t — k) 
P(t) = V2- 35, ho[—k]G(2t — k) 


b(t) = V2- 50, halk] G(2t — k) 


[the time-reverse indexing of the analysis impulse responses with respect to the 
synthesis impulse responses is a characteristic feature of the filter bank formularies.] 


Looking at the DWT formalism in the orthonormal case, we note: For the correct 
functioning of the synthesis filter bank (go, gi), we first need the identities 


(a) p(t) =>, golk — 2nJol™-? (t) 
(b) pk (t) =o, glk — 2nJol” Pa) 


But this is an immediate consequence of the two equations at level m = 0. Then, 
we need a direct sum decomposition 


VrDuiVvyMe wi”) 


where W™) is the closure of the subspace (of yee) which is generated by the 
family (20) pez. 

We insist: The essential point there is that W‘™ is sufficiently large in order 
to allow this decomposition of V°"~); on the other hand, there is no necessity for 
WwW”) to be orthogonal to V“™). 

Let us also point out that the identity V°"-) = VO @ W™ is — under 
our general hypotheses — equivalent to the fact that the 2-translates of go and the 
2-translates of gi constitute together a basis of 1?(Z) (in the Hilbertian sense). 

Concerning the analysis filter bank (ho, hi), our commentaries are the same: for 
a perfect reconstruction filter bank, the couple (ho, hi) can be mirrored to carry 
out the reconstruction (the synthesis) for the inverted filter bank. So we shall have 
similar identities as before for the developments of the translates-dilates of g = £(t) 
and of w = W(t) inside the one-step higher resolution level 


(c) Gu” (t) = DO, holan — KI)" (8) 
(a) dh” (t) =D, balan — KBE") 
as well as the need of a (not necessarily orthogonal) direct sum decomposition 
virD = V™ BW, where W'™ is generated by the family ( DO) rez. 
Arrived at this point, we shall have a perfectly operational DWT, provided that 
the impulse responses (which we suppose finite and of linear phase) satisfy the 
required identities for a perfect reconstruction filter bank. 
But where is the biorthogonality? And what is its interest? 
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Let us give a first answer: The biorthogonality is a luxury which we can allow 
us in the DWT world of finite and symmetric®* impulse responses — this provides 
a great number of very interesting practical examples. And since biorthogonality is 
compatible with our objectives, we shall declare it indispensable. 

Then, the arguments go this way: The essential ingredient of a multi-resolution 
analysis of L?(R) is the more and more faithful discretization of the analog data. 
But: This discretization has to be numerically stable. In conventional mathematical 
terms: the operation which associates with « = x(t) € L?(R) its coordinates (with 
respect to a wavelet basis) has to “respect the norms” (in the orthonormal case, we 
have equality between the norm of « = a(t) € L?(R) and the norm of the series of 
its coordinates in 1°(Z)). In order to guarantee stability, we need at least — in formal 
analogy with the orthonormal case — both for the analysis approximation and for 
the synthesis approximation, developments with respect to the wavelet bases where 
the coefficients are inner products. 

The following observation will serve us as a guide: 

Let us consider the equations for the filtering operations of a perfect reconstruc- 
tion filter bank (in conventional notation): 


yolk] = >7,, [n]ho[2k — n}, 
yilk] = >, e[nJhi [2k — n] 


and 
x(n] = >7, yoll]goln — 21] + 7, wi [lgi[n — 21). 
Put now 
€21[n] = go[n — 21] €21[n] = ho[2l — n] 
e2141[n] = giln — 21] €2141[n] = hy [21 — n] 
wren Wilh] = elm], éaelnl) 


yi lk] = (2[n], €2x41[n)) 
and a[n] = 5%, (x, 21) go[n — 21] + 30, (x, €2141) gi [n — 21] = $0, (x, Em) em[n]. 


All this seems to be a decomposition in a pair of dual bases (€n)nez and (€n)nez 
of 12(Z). And indeed: 


Lemma If (ho, hi) and (go, gi) are the two couples of the analysis filters and of the 
synthesis filters of a perfect reconstruction filter bank, then (en)nez and (En)nez — 
as defined above — constitute a biorthogonal system (a pair of dual bases) in 12(Z). 


Proof The relations of biorthogonality 


( eis 1 ifm=n 
Ri =) gh elge 


are easily verified — cf. the help for the exercise (6) on two channel filter banks). 

We insist: This biorthogonality in 12(Z) does not depend on any particular prop- 
erty of the considered filter bank. 

But let us try to inject it into the continuous world as a twinning condition 
between the discretization of the synthesis approximation (V™) nex and the dis- 
cretization of the analysis approximation (V°™) mez giving rise to a DWT, as de- 
scribed above. 


34 Tn interleaved version! 
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In this situation, we shall call the couple of synthesis and analysis approximations 
biorthogonal provided 


(ye, Gt) =6n1 kLEZ 


for the translates yr = yr(t t) = y(t — k) of the synthesis scaling function y = y(t) 
and for the translates ¢; = ¢1(t) = @(t—1) of the analysis scaling function ¢ = (t). 


Remark Let (v%)xez be the Riesz dual basis — in V — of (yx) kez, guaranteed by 
(MR—5)* . Then yj, is the orthogonal projection of ¢, onto V©), for all k € Z. 
We see that V© and V©) are altogether rather engaged. 


The target consequence of the biorthogonality on the level of the scaling functions 
is the biorthogonality of the (synthesis and analysis) wavelet bases: 


Proposition The biorthogonality of the scaling functions (pn, Gn’) = Onin, n € Z 
implies the biorthogonality of the associated wavelet bases 


(WO PO) = Simm Ske mm’, kk! EZ 


Proof We shall reason by means of a geometric constellation. In order to establish 
it, we need the following 


Lemma With the hypotheses of the proposition, we have the following identities: 
(1) EP EOP \=Siar man! eZ 

(2) WE, DO) = San myn! EZ 

3 COS wo”, @ a =0 mnn eZ 

(3) (on™, n, 


Proof (1) Immediate (change of variable in the considered integral). 


(2) 
fm = Do alk— ante" 


PAGO =n gon) 


(bk, PP) Sige 2n]ha[2n’ — Yipe"-Y, gf") 
k,l 


= S5 gilk — 2n\hi[2n! — bi] 


k 
= (e2n+1[K], €2n741[K]) _ Onn! 


(Ph, BE) = S° golk — 2njhal2n’ — (eer, af") 
k,l 


= S5 go[k — 2n\ha[2n! — k] = (ean[h], €anr+[A]) = 0 


(HL, BL) = (ean+i[k], Ean’ [k]) = 0 
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Observation The biorthogonality of the scaling functions (pn, Gn’) = Snint 
n,n’ €Z implies the perfect reconstruction for the associated filter bank (which 
is equivalent to the biorthogonality  (en,@n') = Onn (in 12(Z)) of the system of 
the synthesis translates (e,) and of the system of the analysis translates (€,,/ )). 


The argument: First, we have: (a en) = dri, k,l € Z. Then, write 


Qn = S- aolk Injy?, 
k 


Qn! = x ho[2n' = oi. 
1 


Hence, via inner products: 


Snn! = S— golk — 2nJho[2n! — I]5x,. = S~ golk — 2n]ho[2n' — kj. 


kyl k 


But this is the condition of perfect reconstruction. 


But let us return to the proof of the proposition. According to the identities 
(3) of the lemma we have, for all m € Z: W‘” is orthogonal to V°™, and W‘™ 
is orthogonal to V’™. Now, the V“” as well as the V°™ constitute a chain of 
closed subspaces, and every v(™ contains all W“™ for m’ > m; equally, every 
V™ contains all W°” for m! > m. 

Finally; W°™ and Ww are orthogonal for m 4 m’. But this is precisely — 
with the point (2) of the lemma — the claim of the proposition. 


Remark We note how the biorthogonality (in 12(Z)) of the impulse response 
2-translates — which always takes place whenever we dispose of a perfect recon- 
struction filter bank — serves to propagate the biorthogonality from the level of the 
scaling functions to the level of the associated wavelet bases. 


Consequence Under the hypotheses above, every x = x(t) € L?(R) admits two 
“dual” developments 


2 ae a ea ee 
k,meZ k,meZ 


We have finally got the desired coordinatization, but not yet completely: the situa- 
tion effectively aimed at is that of a pair of Riesz bases for L? (R). 
In other words: We need two positive constants A and A with 


1 jen P 
Fla? < So |e, dk) Ps Aljal?, 
k,m 


a m 
Slo? < So | (e, 8) Ps Alla? 
k,m 


Let us try to well understand the problem: A priori, the two Riesz bases 
(y(t — k))xez and (¢(t — k))xez are “externally dual” on resolution level 0 (cf. a 
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previous remark). This gives the same situation for (wo) nen and for (bo nex 
on a given resolution level m. The problem is to make sure that the “local” Riesz 
constants do not explode at infinity (in m). 

Let us repeat: It is the uniformity of the Riesz bounds — across all the resolution 
levels m — at which we aim. This also implies that (be Ji, mez and (be Jie mez 
will then be unconditional bases (i.e. that the summation mode will not affect the 
results of the considered developments). 


Design Procedures 


In this last section, we shall try to answer two questions which naturally arise in 
application-oriented spirits: 


(1) What is the practical interest of the fact that a (perfect reconstruction) filter 
bank is the DWT associated with a multi-resolution analysis (eventually split 
into analysis copy and into synthesis copy, with biorthogonal coupling)? 

(2) Which are the criteria that produce realizable and practically interesting Dis- 
crete Wavelets Transforms? 


On a very concrete level of reasoning: How can we justify the choice of the DWT 
5/3 spline and of the DWT 9/7 CDF for JPEG 2000? 

Our answers will follow — at least in certain technical details — a logic of “spectral 
arguments”. Hence we have to pave the way. 


The Frequential Formulary 


Recall briefly the characteristics of a (biorthogonal) DWT: 


(1) The formulary of the (perfect reconstruction) filter bank: 
The perfect reconstruction condition: 
(R) YO, holklgol2n — k] = {) a 
[Did we normalize? — cf. exercise (1) at the end of this paragraph.] 
The high-pass impulse responses (derived from the low-pass impulse re- 
sponses®” ): 


hi[n] = ha[n — 1] = (—1)"go[n], 


gin] = mln + 1] = (-1)"holn). 
Caution: On this formal level, there is no (discrete) time-inversion between 
analysis and synthesis. The inversion shall take place in the “operational” for- 
mulas. 
We suppose tacitly that all impulse responses are of finite (odd) length and 
symmetric. Our notation will altogether respect the signs (in discrete time) 
where it is demanded by general theory. 


35 We also shall have the other viewpoint: The synthesis impulse responses derived 
from the analysis impulse responses. 
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(2) The scaling equations: 
y(t) = V2- >), golk]p(2t — k) 
(S) 
P(t) = V2- 7, ho[—k] G(2t — k) 
The “high-pass” equations concerning the mother wavelets )(t) and )(t) (and 
making appear gi = (gi[k]) and hi = (hi[k])) are analogous and logically 
secondary. 
(3) The condition of biorthogonality: The condition of biorthogonality (ym, Gn) = 
dm,n on the integer translates of the two (synthesis and analysis) scaling func- 
tions has no discrete equivalent. 


We saw previously that it implies the condition of perfect reconstruction (R) which 
is equivalent to the biorthogonality (em,@n) = dm (in 12(Z)) of the system of 
synthesis translates (e,,) and of the system of analysis translates (€,): 


€21(n] = go [n = 21] €21[n] = ho [21 = n] 


eai+i[n] = giln — 21] Eai4i[n] = hi[2l — n] 


Now let us pass to the reformulation of our characteristic identities in “frequen- 
tial notation”. 


Recall: The Fourier transform = f(w) of £ = (f[k])kez — which we always 
suppose of finite support — is the trigonometric polynomial 


Two important values: 


fo) =>— fia] 


k 


fr) = $0) * Fla 


k 


Let us begin with some usual normalized notation: 


mo(w) = Fy Go(w) 


mi(w) = 2yGt(w) = AgeGr(w) (since gf fn] = giln + 1)) 
mMo(w) = Jz ho(—w) (we have transformed (ho[—7])nez) 
mi(w) = ahi ( w) = 5 e” ha (—w) (since hi[—n] = hi[—n — 1)) 
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Our “braid” relations 


Mo(w) = mi(m — w) and mi(w) = mo(m — w). 
Finally, there are “boundary values” 

mo(0) = mo(0) =1 mo(7) = mo(7) =0 

mi(m) = mi(r) =1 mi(0) = m1(0) = 0 


the reason for which will become apparent in a moment. 
Now let us move on to our program of translating the formulary. 


(1) The perfect reconstruction: 


Write mo(w) = Ws : oy go[k]e7*#36 


mmo(w)Mis(w) = 5 ( s nit ei 
n k+l=n 
mo(w + 7)Mo(w +7) = ( S- (igh nit eine 
n k+l=n 


Hence: 
mo(w)ing(w) + mo(w + m)mg(w +7) =>, (Do, golk|ho[2n — k]) ei?” 


3° We shall use ()* for conjugation, in order to avoid a bar-invasion above the for- 
mulas. 
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Finally?’: 


Mo(w)mg(w) + mo(w + 7)mMg(w +7) =1 
—= 


(R) yee go|k]ho [2n — k] => 6n,0 


Remark So we have found a frequential formulation of the perfect recon- 
struction property. This means that we have characterized the biorthogonality 
(€m,€n) = dmjn of the systems of synthesis vectors and of analysis vectors (in 
12(Z)) associated with the considered filter bank. 


We shall remain in the setting of perfect reconstruction filter banks. 
Put w = 0. Then mo(0)179(0) + mo(m)mo(m) = 1. Now, we face a surprising 
intervention of the analog world: 


Test of compliance. If the considered filter bank is a DWT associated with the 
couple of scaling functions (y,%) — which we suppose to only admit a finite 
number of discontinuities — then we have necessarily 


(ie. 9, (—1)* go[k] = 32, (—1)*ho[—k] = 0) [without proof]. 


Hence, we get the following (normalized) “boundary conditions”: 


mo(r) =0 mo(m) =0 mi(m) = 1 mi(m) =1 
weak, low-pass weak high-pass 


(2) The scaling equations: The scaling equations (S) become 


wor=m(3)-9(9) 


37 You are right: we work with a = det A = 1 (A = the analysis matrix). For certain 
applications, this is not the best choice. 
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By iteration we obtain®® 


lw) = TT, mo ($5) 
Sw) = TT, to (3) 


(P) 


These identities express G(w) and ¢(w) as infinite products of trigonometric 
polynomials. 

We note: The constituents of the infinite products only depend on the filter bank, 
i.e. only depend on go = (go[k]) and on ho = (ho[k]). 


Conclusion The formulas (P) allow the (re)construction of y(t) and of £(t) 
from go = (go[k]) and of ho = (ho[k]). 


We have hence a |test| whether a given filter bank is indeed a Discrete Wavelet 
Transform, i.e. if it comes from a multi-resolution analysis — copy synthesis, 
copy analysis: 
We must guarantee that the infinite products of the trigonometric polynomials 
converge properly (@(w) and G(w) must be square integrable). 
We shall take up this theme in a moment. 

(3) The biorthogonality: The condition of biorthogonality (on the integer translates 
of the two scaling functions y = y(t) and ¢ = ¢(t)) does not translate into 
a “frequential identity” deduced from the impulse responses of the associated 
DWT. 


Altogether, we dispose of a link by means of the Poisson formula. 


Proposition The biorthogonality condition (Yn, Pn’) = Sn.n’ ts equivalent to the 
summation formula 

S "Gv + 2kr)G(w + 2kr)* = 1. 

keZ 


Proof We always shall suppose y = g(t) and ¢ 


grable. Consider the trigonometric series S(w) 
yO); p(Eje**§)e** and the 2-periodic function T(w) = eee 
Qkr)G(w + 2kr)*. 

The biorthogonality is equivalent to S(w) = 1, and the validity of our summation 
formula is equivalent to T(w) = 1. Hence we have to show that S(w) = T(w). 


The fundamental auxiliary result is the following: 


Lemma Let F(w) be a summable function such that G(w) = \o pez F(w + 2k) is 
square integrable on [0,27] (G(w) is 27-periodic). 


Then: GWw)= 3 Vorez F(—k)e iF, 


38 This is true up to a multiplicative constant, which will be usually equal to 1. 
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Proof By our hypotheses, G(w) admits a development in a Fourier series. Let cz, be 
its kth Fourier coefficient; then 


Ck = as G(w)el*’ dw 


But: (ie G(w)e 8" dw = fs F(w)e"®"dw = F(€) 


Hence: cp = + F(-k). 


Let us return to the identity S(w) = T(w). 


We shall apply the foregoing lemma, with F(w) = ¢(w)¢* (w). F(w) is summable, 
since y = y(t) and ¢ = Gt) are supposed to be square integrable. 

We have: T(w) = G(w), that’s already promising. We ae a the iden- 
tification of (9(€), (Je). But ((6). ae) = [ OS" (GeMae = 
2 f F@ eas = LF(-K), 


Exercises 


(1) a=landa= 34.9 
Recall: Accept first a € R* Hanes Then the condition of perfect recon- 
struction will be (R, ) >, Polk] (— )¥ hs [2n — k] = a@- bn,o supplemented by the 
identities 


(—1)*Ai Ik], 
(—1)"holk}.- 


Note that the variant for the condition of perfect reconstruction 


ho|k]go[2n — k] = bn 
(R) DY holk]go[2n — k] = dn,0 
is independent of the value of a. 
(a) Show that we have: a = 3(h§(O)Ai(t) + hi (O)A}(m)) 
t 
1 


(b) Show that we have: eae + hi (1) 9h () = 2 


(independently of the value a = det A !) 
[In the case of a filter bank which is a DWT, we thus have 


a=hA,(O)Ai(m) +1=395(0)Gi(7) and — hf(0)95(0) = 2 


In other words, one will put — in theory — usually 5°, go[k] = 
3, holk] = v2] 
(c) Now let us pass to a frequent renormalization: 
We shall replace ho by hi) = Fi ho (hence we shall take (the coefficients 
of) ™o(w) as low-pass analysis filter). 


Show that we have necessarily: 


39 q = det A = the determinant of the analysis matrix for the considered filter bank. 
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Ne oo = V2- go (we will get 2- mo(w) as low-pass synthesis filter). 


(ii) oi ny | [k]} =1 and ae gS [k] = 2. 
(iii) a= 4. 


(2) Consider mo(w) = — ge" + fe” + 2+ Fe! soe 
(a) Verify that mo(0) = 1, mo() =0 
(b) Find mo(w), symmetric (as mo(w)) and of length 7, such that 


Mo(w)mg(w) + mo(w + 1)mg(w +7) = 1. 
[Help: You have to resolve a linear system which stems from the conditions of 
perfect reconstruction, with the supplementary constraint ™mo(7) = 0. 
So, you will deal with the 4 x 4 system coming up from the three equations 
yop a [2n — k]ho[k] = 6n,0, for n = 0,1, 2, together with the additional condition 


(- )¥ho[k] = 0. 
Caution: The summation index k varies from —3 to +3, although you will have 
— by the postulated symmetry — only four unknowns ho[0], ho[1], ho[2], ho[3)].] 


Solution: First, we obtain — with x, = ga holk] — the following linear system: 


6 1 = 
5r0 + ©1 — 3X2 ele 
1 

— tx, + x3 =0, 
1 12 
sto + @1 3%2 + 23 =0, 
xo 201 2x2 223 0. 


This gives finally: 


3 ew 3 eZiv 73 el” 17 73 iw 3 a ae 3 ew 
280 56 280 56 280 ‘ 


(3) (QMF filter banks)*° Consider a (finite support) filter bank which is “self- 
orthogonal” in the following sense: 


holk] = go[—kl, 
halk] = gi[—k] = (-1)"**ho[-(k + 1)]. 


(a) Put mo(w) = 5 G0(w). 
Show that the condition of perfect reconstruction is equivalent to 


mo (w) 


| mo(w) |? + | mo(w +7) |?=1. 


(b) Now suppose the reconstruction condition (a) and the normalization 
mo(0) = 1 valid (one speaks then of a QMF filter bank). Suppose moreover 
that 6(w) = []2, mo ($) is a square integrable function. 


Show that then our filter bank is the DWT associated to a multi-resolution 
analysis with scaling function y= y(t) <> ee | 6(w + 2kz) = 1. 


(c) Now pass from mo(w) to mi) (w) = Mmo(3w). 


40 QMF = Quadratic Mirror Filter. 
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Show that the filter bank defined by m®) (w) (i.e. by up-sampling to the order 
3) is still a QMF filter bank (if the filter bank defined by mo(w) is one), but, 
on the other hand, that it never will be a DWT (if the filter bank defined by 
mo(w) is one). 

(This answers the question (b) of the exercise (2) in the section “The Ortho- 
normal Situation” ). 


The Filter Banks Which are Discrete Wavelet Transforms 


The initial situation. A perfect reconstruction filter bank: 


Zo = (go[k]) the low-pass synthesis impulse response. 
ho = (ho[k]) the low-pass analysis impulse response. 


(The high-pass impulse responses g; and h, will then be fixed). We suppose go and 
ho to be of finite (odd) length and symmetric. Consider 


1 —ikw 
mo(w) = B Date 


~ 1 —ikw 
mo(w) = Fi d, ho[—k]e 


and 
sw) = [Tm (2). 
at) =[] (5): 


In order to obtain that the considered filter bank is the DWT associated with a 
coupled multi-resolution analysis, with scaling functions (y, ¢) (preferably biorthog- 
onal), we need (at least) that the infinite products of the trigonometric polynomials 
above are convergent and that y = y(t) and ¢ = £(t) are square integrable. 

How can we find the appropriate conditions on mo(w) and mo(w) such that 
y = y(t) and ¢ = G(t) will be validated? 

Recall our first criterion of compliance: 

If the considered filter bank is indeed a DWT, then we necessarily shall have: 
mo(m) = mo(m) = 0. Hence, only low-pass impulse responses go and ho with 
>, (-1)* 90[k] = 32 ,,(—1)*ho[k] = 0 will be accepted. 


Write then: ms 
1 4 Qriw 
mo(w) = (43 ) po(w), 


N = the multiplicity of the root 7 of mo(w) 


mote) = (72S) pote, 


N = the multiplicity of the root 7 of mo(w) 


with po(0) = po(0) = 1. 


We search for a sufficiently general result which guarantees the realization of a 
filter bank as a DWT. 
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Let us begin with an observation which is indispensable for the understanding 
of the technical hypotheses which control the central theorem of this section. 


Consider 
1te\* 
mMo(w) = (+=) po(w). 


Ate = c0s (4) | 
2 2 


Now, 


and 


We hence obtain for ¢(w) = [7 mo (<): 


| ew) I< C+ |e)” TT | po) | 


k=1 


In order to guarantee by simple means that @ € L?(R), we need only that 
Tz, | po (+) | is bounded by (1+ | w |)’, where N —b > 3. 

The realization of this program demands some technical vigilance. Hence con- 
sider B = sup,,cio,2n] | Po(w) | and put 6 = Log,B. Then we obtain the desired 
estimation. 


Lemma | ¢(w) |< C- (14+ |w [joer=®) 


Proof It remains to show that []*~, | po (=) I< c’- (1+ |w)?. 
First, po(w) is a trigonometric polynomial, with po(0) = 1. Consequently 


| po(w) |S 1telw |< etl. 
Put q(w) =] T=, po (ge) 


Ww clw 
| gw) |< exp(c Lane ell, 


| q(w) | is bounded for | w |< 1. Let us then find an estimation for | w |> 1. 
Let nu € N with 2"#7! <|w |< 2. 


| a) | 


4 w ~ 27>"! Ww 4 w é Aigtin 
IL |» (=) |- [| x0 (A) I< [] |» (=) | -e° < e°B™ 
k=1 k=1 k=1 


= ef gnwlog2B < e° (2 | w ees = eB | e [Pree Pas Cc’ : (1+ | ay \)° 


Remark We can give a more refined version of the lemma above, when replacing 
the exponent b by the critical exponent 3 of the trigonometric polynomial mo(w). 2 
is defined as follows: 
First: Bo = 1, By =sup, er | [],_, po(2*w) |, j > 0. Then: b; = Log; B;, j > 0 
Finally: 8 = infjs0 b;. 


We note that b; is the exponent b of the foregoing lemma. 
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Now let us pass to the principal result of this paragraph. 


Theorem Let us start with a perfect reconstruction filter bank. 


—iw N iw N 
Associate with mo(w) = (45=) po(w) and with mo(w) = (4s=) Bo(w) 
the exponents b and b (the critical exponents 3 and B). 


(1) IfN-b>4 and N-b>4 (N-B> 4 and N— > 4), then p = y(t) and 
~ = P(t) are square integrable. More precisely: 


(D) | Bw) | + | Bw) |< C- (At |w [727 


with appropriatee>0. 

(2) If (D) holds, then the families (pp = p(t—k))xez and (Gx = G(t—k)) kez are nec- 
essarily biorthogonal and generate two biorthogonal wavelet bases (Oe a ied 
and Gl” Vin nee. : 


Proof (1) We already treated the question. 

(2) It is the inevitable biorthogonality which fascinates us. Let us show then how 
the (D)-controlled decreasing of ¢(w) and of G(w) imply the biorthogonality 
(Ges Per) = Sk,kr 
Thus consider the “natural” approximations of ¢(w) = LE Mo (x) and of 

gw) = TTR, ™Mo(S), given by 


ho(w) — ho(w) => T[-x,x]> 


in (w) — 1G ieee Mo (+) : T[_on p27 rr] and by hn (w) = Lis Mo (= 
n> 1. 
First, let us show (by recursion) that (hn(t — k), hn(t — k’)) = bg, 47 


> 


) y T-angjon-]; 


= 2" ‘ TIm (2*w) rng (2*w) ) el?’ d 
ae 
=o / = (Te 2%) [mo(w) rng (w) 


+mo(w +m) mg (w + 1)) "dw 


v n-2 
=2r1 / (i notatonsca's)) ee day 
—7™ \k=0 
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But: fe ho(w)ho(w)*e"’dw = ime e"'’ dw = 2760,. (We observe the crucial in- 
tervention of the perfect reconstruction identity.) 


The remainder is easy: It is clear that hn(w) and hn(w) pointwise converge to- 
wards G(w) and ¢(w), respectively. Moreover, by the hypotheses of the theorem, 
the functions hy, and hn also verify the inequality (D) for their Fourier trans- 
forms. We only have to apply the theorem on dominated convergence in order 
to conclude that the sequences hn and hy tend to y and ¢ in L?(R). This finally 
gives the biorthogonality of the sequences (yx)kez and (Px)rez- 


Remark Let us insist upon a surprising feature of the theorem: If you want to 
design a DWT, starting with a candidate filter bank, and passing by the scaling 
equations in spectral form, then — by any natural mathematical approach — you 
hardly can avoid the condition (D) as a guaranty for valid scaling functions — and you 
will necessarily arrive at biorthogonal wavelet bases. All these things are interesting, 
but maybe too abstract for an intuitive and practical spirit. Hence we eagerly should 
present an (almost) playful aspect of the affair: 


How to visualize » = y(t) from go = (go|k]) ? 


We shall be confronted with the 


Subdivision Algorithm 


Recall y(t) = V2: Sh go[k]y(2t — k)*" 

Hence: y = ¢(t) is a fixed point for the operator (T'f)(t) = V2->-, go[k] f (2t—k). 
Let us then try an approximate construction of » = y(t) by iteration. We shall begin 
with uo = T_3,4] and shall have un =T”"(uo), n> 1. 

Observation The operator T transforms a step function (a piecewise constant 
function) into a step function — by dyadic subdivision of the step intervals. 

In our case, Un will hence be constant on intervals of length a more precisely 


on the intervals of type ]2~” (k — $) ,27" (K+ 4) [. 


Exercises 


(1) Let &,(w) be the Fourier transform of uy, = un(t). Show that 
fin(w) = tes mo(2~*w)tio (27 w) 
[Help: Verify first that (T f)(w) = mo (4) f($)I 
Associate with un = un(t) the sequence s,, of its values on the various intervals 
J2-"(k — 5),2°"(k + 5) [snl] = un(2-"k)]. 
The step function uy, is completely described by the sequence s,,. 


41 We tacitly shall suppose go to be of finite (odd) length and symmetric. 
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(2) Let S,,(w) be the Fourier transform of the sequence s,,, considered as an element 
of the vector space I? (srZ) a 
Show that 
So(w) = d' 
Si(w) => 2mo ($) 
More generally: 
Sn(w) = 2” - TTP _, mo(27*w) 
ie. that Sn4i(w) = 25n(w)mo (seer). 


(3) (The subdivision algorithm) Show that 
sol[k] = 5x,0, 
si[k] = V2- go[k], 
sneilk] = V2) golk — 2]sn[l). 


Accept the following result: If g = y(t) is the valid scaling function of a (synthesis) 
multi-resolution analysis, then the subdivision algorithm converges uniformly to 
p = v(t). 

Note the important aspect of graphic visualization of yp = y(t) from go = (go[k]). 
In other words: In a first approach, the validity of y = y(t) (and its degree of 
regularity) are visually verified. 


(4) We now insist on the symmetry of go: go[—k] = go|k]. Show that then s,[—k] = 
8n[k] for k,n > 0 (and that y = y(t) is symmetric: y(—t) = y(t)). 


(5) Show, in the situation of the preceding exercise: If the support of go is between 
—N and N, then the same is true for un = un(t), m > 1, hence also for 


p= v(t). 


(6) Consider mo(w) = fe” + 4 + te™™. 


So we have: go[0] = en go[1] = go[—1] = = 
(a) Compute so, $1, S2 and s3. 
(b) Show, by recursion: s,[k] =1-4 O0<k <2". 
(c) Deduce: 
1+t -1<t<0, 
p)=s1-t o<t<1, 
0 else 
(7) Now consider no(w) = — ge” + ze” + $+ Ge” — ge 7 
Let us sum up: ho[0] = 2¥?, ho[1] = ho[-1] = ¥2 ho [2] = ho[—2] = — 2. 


Compute 8, 81, 82 and 83 by the subdivision algorithm for ho.*? 


Trace the step function described by s3. 


Commentary The exercises (6) and (7) are meant to give an idea of the shape 
of the scaling functions y = y(t) and ¢ = G(t) “behind” the DWT 5/3 spline. 


” Pay attention to fractionary counting: For a recursive argumentation, we have to 
keep track of the dyadic sampling refinement. 
43 The symmetry of ho will hide the time-reversed counting “on the analysis side”. 
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(8) 


-1 -05 0 05 4 


$1, S2, $3, for go of the DWT 5/3 spline. 


Let y = y(t) be a valid scaling function — which we have approximated by the 
subdivision algorithm, beginning with go = (go[k]). 

Change the viewpoint: Carry out a “frequential” approximation of the form 
On(w) = [Tay 0(27 "wan 20] 

Gn(w) —> @(w) (in L?(R) — take this for granted). 

Put yn(t) = ao ae Ls mo(27*w)e'™* dw 

n(t) —> y(t) (this is immediate). 

Show that the subdivision algorithm interpolates at the same time the sequences 
of functions Un = Un(t) and Yn = Yn(t): 

$n[k] = Un (+) = Pn ia 

[Help: On the one hand, |]j_, mo(2~*w) = sz - 0, Sn [k]eizre on the other 
hand 
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8, for ho of the DWT 5/3 spline. 


= = ™ 1 k jek gy 
[[ me *w)I[-2n,2%m] = Pn(w) = an S- Pn (=) eT ane, ann] 
k=1 
(Fourier transform of the Shannon interpolator)] 
Complement: The Subdivision Algorithm for w = w(t) 
The identity w(t) = V2- 3°, m[k]y(2t — k) becomes, in frequential notation 


sey ettm (3) 9(G) <4 (8) To (B). 
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S2 for ho of the DWT 5/3 spline. 


This leads us to consider the following recursive algorithm: 
so[k] = dk,0, 
si [k] = V2- gi fk], 
sn4ilk] = V2-S~ go[k — 2s (0). 
1 
(9) Let S/,(w) be the Fourier transform of s}, € 1° (s2Z) n> 0: 


Siw) = S— sulkle te’. 


k 
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83 for ho of the DWT 5/3 spline. 


Show that 


(10) Let us define an approximation 7, (t) —> 7(t) by frequential truncation: 


Wn (w) =e im, (g) . ITi_ mo (=) - Tj-anz,2»7] (the convergence arguments 


are the same as in the case of yn(t) —> y(t 


Show that s’,[k] = vn (+) keEZ, n> 0. (cf. the aid of exercise (8)) 
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Construction of Biorthogonal Wavelet Bases 


The star construction for biorthogonal wavelet bases comes from A.Cohen, 
I.Daubechies and J.-C.Feauveau. 

We shall restrict ourselves to the case where the impulse responses are of finite 
(odd) length and of linear phase (this means that the interleaved notations are 
symmetric with respect to n = 0). 

mo(w) and 7o(w) will then be real polynomials** in cos w. 


to ctw \ NV 
Recall { °* +27 — = (cos aye = (Ga + cosw)) 


yz 


is a polynomial in cosw if 


and only if N is even. 


Hence, in our case, mo(w) and mo(w) must have a root of even multiplicity at w = 1: 


mo(w) = (cos i. - 7 (cosw) 


x ab 
Mo(w) = (cos 2) - F(cosw) 
with L,L > 1, ie. N,N > 2. Put now M = L+L, R(x) = r(x)F(x). Then the 


perfect reconstruction condition (R) becomes 


(cos? =)" R(cos w) + (cos” TEMP ensled +7))=1. 
Observing that sin? = 4(1 — cosw), and putting (logically) R(x) = 


Q Gd _ a) we obtain the identity 


(cos? S)™"-Q (sin? 2) (co 27)" .Q (sin? +7) i ie 


2 2 2 
Observation (I.Daubechies) The polynomial identity u“Q(1 — u) + 
(1 — u)“Q(u) = 1 admits a unique solution of minimal degree M — 1, which 


is Qu) = Dpno Cu". 

So, we can (already) pass to a short summary: 

At a first approach, the design of biorthogonal wavelets with associated DWT 
of finite support and linear phase, faces the following situation: 

We shall fix the multiplicities N = 2L and N = 2L for the root w = 7 of the 
trigonometric polynomials mo(w) and ™mo(w). Then, we shall consider 
R(x) = oy (Mt) a2)”. MaLt Eh. 


n=0 n 2” 


Choosing a factorization R(x) = r(x)f(x), we obtain 


2L 
mo(w) = (cos #) -7r(cosw) = Ww >, golkle*”, 


Mo(w) = (cos aes - F(cosw) = w >, ho[-k]e7*”. 


44 Recall that our filter banks are real. 
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All this seems to be a good beginning. But there remain two fundamental questions: 


(1) Do the equations 
ate)=[]m(e) $0) =[] (52) 


define valid scaling functions y = y(t) and ¢ = @(t), ie. which are square 
integrable and give rise to biorthogonal wavelet bases ? 

(2) Which are the criteria for the choice of the multiplicities N and N and for an 
appropriate factorization of R(x) into r(x) and F(x)? 


Let us consider the first point. The basic result will be stated in a surprising 
(and even enigmatic) way. Hence, we have to prepare the field. 


Exercises 


We consider a coupled multi-resolution analysis’? of L?(R), with the synthesis scal- 
ing function y = y(t) and the analysis scaling function ¢ = ¢(t). 
(1) Show the following properties: 


(a) If the support of go = (go[k])xez is between N; and N2 and the support of 
ho = (ho[k])xez is between N, and No, then the supports of y = y(t) and 
of ¢ = G(t) are in the intervals [N1, No] and [N1, No], respectively. 

(b) If go and ho (have an odd number of non-zero coefficients and) are sym- 
metric with respect to k = 0, then y = y(t) and ¢ = ¢G(t) are symmetric 
with respect to t = 0. 

(2) Now associate with y = y(t) and ¢ = ¢(t) the two functions 


ow) = S° | GWw+2rk) ? and aw) = S° | Gw+2nk)?, 
kez kez 

(which are real and strictly positive). Show that 

a(w) = 39, (v(t), et — ke", 
&i(w) = D7, (P(t), Bt — ke. 

[Hence we deal with strictly positive real trigonometric polynomials — provided 

Zo and ho are real, of odd length and symmetric with respect to n = 0] 

Help: You should use the lemma which served to establish the summation for- 

mula characterizing biorthogonality — now with F(w) =| ¢(w) |’. 

(3) (The point) - 

Consider the operators Po and Po on trigonometric polynomials, defined by 
(Pof)(w) =| mo (3) PF (3) +l mo(F +m) PF (G47), 
(Pof)(w) =| tito (3) PF (G) + | 0 ($ +7) PF (G+7)- 

Show that the polynomial a(w) is a fixed point for the operator Po, and that 

the polynomial @(w) is un fixed point for the operator Po. 


45 We know what is meant. 
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Help: Try to really understand the equalities 


(Poa)(w) =| mo ($) [916 ($ + 2en) I? 
+| mo (S+7) Pole (S++ 2%) [? 


= 5° | wt 4kr) P +5 > | Sw +20 + 4kr) P= aw). 


keZ keZ 


Remark In case of a biorthogonal coupled multi-resolution analysis of L?(R), we 
can show that a(w) and G(w) are, up to a non-zero scalar multiple, the only fixed 
points of the operator Po and Po, respectively. All this is (mathematically) very 
satisfactory. But for a practical spirit, it seems to be rather academic stuff. So it is 
really surprising that the following fundamental theorem — the statement of which 
is highly arid — actually yields a simple and efficient test which validates a perfect 
reconstruction filter bank as a DWT associated with a couple of biorthogonal wavelet 
bases. 

So, let us state one of the principal results of the theory of biorthogonal wavelet 
bases, which provides — despite its theoretical look — the indispensable practical tool 
for an efficient resolution to our problem: How to sort the filter banks which are the 
candidates for “biorthogonal wavelet transforms” ? 


Theorem (Cohen—Daubechies—Feauveau) Fixed Point Criterion. 

Let go = (go[k]) and ho = (ho[k]) be two real finite support sequences (eventually 
symmetric with respect to k = 0). 

Suppose that the perfect reconstruction condition (R) holds. 

Let us define the trigonometric polynomials mo(w), mo(w) by their usual formu- 
las from go and ho, and ¢ = ¢(w) and ¢ = ¢(w) by their product formulas. 

Then the following two statements are equivalent: 


(1) y= y(t) and ¢ = G(t) are square integrable, and (y(t — k), G(t —1)) = bx.1*® 

(2) There exist two strictly positive (real) trigonometric polynomials fo and fp such 
that Po fo = fo and Po fo = fo, and fo and fo are unique (up to a non-zero scalar 
multiple). 


[Without proof] 

Note: The property of the unique fixed point (in the projective sense) for the two 
operators Po and Pp is characteristic for the existence of valid scaling functions y 
and ¢ as well as for the biorthogonality of their Z — translates. 

What is important for the designer — as we already underlined — is that this 
austere theorem actually gives a test whether two candidates go and ho (satisfying 
the condition of perfect reconstruction (R)) define the two low-pass filters of a DWT 
associated with a couple of biorthogonal wavelet bases. In detail: 


Observations concerning the compliance test for go = (go[k])*”: 


46 Recall that the variable t is bound — by integration — and that the two integers k 
and / are free. 
47 For ho = (ho[k]) the situation is similar. 
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(1) The terms | mo(w) |? and | mo(w + 7) |? are modulation invariant; thus we can 
suppose that 


a ~ —ikw 
mow) = Fe d_wltle 


and consequently, | mo(w) |? and | mo(w +7) |? will be trigonometric polyno- 
mials of degree < N. 

(2) If fo = fo(w) is a trigonometric polynomial such that Po fo = fo, then we have 
necessarily fo(w) = pears folkle*”, ie. fo = fo(w) is a trigonometric 
polynomial of degree < N. 

(3) Let then Ey be the vector space of the trigonometric polynomials of degree 
<N. 

Ew is stable under the operator Pp. 
Let Mo be the matrix of Po relative to the basis {e-*”, —N < k < N}. Then 


Mo = (gmn)—n<mn<n With gmn = > golk]go[k +n — 2m] 
k 


(cf. the exercises below). 

(4) The condition of the (strictly positive) “unique” fixed point of the operator Pp 
now is verified as follows: 

(a) Triangulation of the 2N +1 x 2N +1 matrix Mo — Ten +1 in order to make 
sure that Mo admits the value 1 as a simple eigenvalue. 

(b) In the affirmative case, we now dispose of a triangular system of 2N equa- 
tions in 2N + 1 unknowns in order to determine “the” eigenvector which is 
fixed for the action of Mo. 

(c) Let fo(w) be the (real) trigonometric polynomial the coefficients of which 
(in complex notation) are the components of this eigenvector. Remains to 
verify that fo(w) is strictly positive. 


Exercises 


(1) (Matrix representation of the operator Po). Let us fix the notation: 


We will search the coordinates ((Po f)[m]) of the complex notation of the trigono- 
metric polynomial Pof in function of the coordinates (f[n]) of the complex 
notation of the trigonometric polynomial f** 


48 The increasing order will be that of the “tap values”, i.e. of the powers of X = 
ene: 
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(a) Show that 


me (5) mo (5) t (5) = : ys OOO! go[k] go(k-+n—2m)) f[n])e"”. 


meE4Z n k 


(b) Show that 
mo (Stn) ma (S40) F(F +7) 
-3 bor (= > ee) 7) cae 
(c) Show that 
[mo (3) Fr(3) +1 mo (S +n) Ps (S47) 
ae b> eS nikinit-+ 2m) sn) ad 


meZ n 


(d) Deduce the matrix representation of the operator Po: Suppose that the 
support of go is between 0 and N. Then the transformation of the co- 
ordinates of f(w) = Sa f[nje"’ into coordinates of (Pof)(w) = 


a (Pof)[mje~*"” is done by the following matrix multiplication: 


m=—N 
Pofl-N] 9-N,-N  9-N,-N4+1 ++» 9-N.N hi eral 
Pof[-N +1] 9-N+1,-N 9-Nn41,-N41 --. g-Ntin |{ f[-N+]] 
Pof[N] QN.-N  9N,-N+1_ ++» 9N.N F{N] 
where gmn = >>, go[k]go[k +n — 2m] for —N<m,n<QN. 
(2) (Matrix representation of the operator Pp). 
1 —ikw 
mo(w) = Va : S > hol-A] k f 


(Pof)(w) =| tho (F) Pt (S) +o (S +2) Pe (G +2) =doPos lime. 


Show that 


(Pof)im] = >> (= ho[—k]ho[2m — n — s) (nl. 


n 
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(3) Let go = (g0[0], go[1], go [2]) have support {0, 1, 2}. 


Put a = go[0]go[2], 6 = go[0]go[1] + go[1]90[2], ¢ = go[0]? + go[1]? + go[2]?. 
Show that the matrix representation of the operator Pp on the trigonometric 
polynomials of degree < 2 is given by 


(Pof)[-2] a0000 f[-2] 
(Pof)[-1] cbal0 f[-]] 
(Pof)[0] | =| abcba f(0 
(Pof) [4] 00abe f(l] 
(Po f)|2] 0000a Ff [2] 


Let ho = (ho[0], ho[1], ho[2]) have support {0,1,2}. Put a = ho[O]ho[2], 
b = ho[0Jho[1] + ho[1]ho[2], ¢ = Ao[0]? + Ao[1]? + o[2]?._ 

Show that the matrix representation of the operator Po on the trigonometric 
polynomials of degree < 2 is given by 


(Po f)[-2] a0000\ /f[-2] 
(Pof)[-1] cba00) { f-1] 
(Pf) |=]|abcbal | flo) 
(Po f)(1] 00abe f(] 
(Pof) [2] 00000) \ FQ] 


(Attention This is a consequence of the exercise (3); but by what argument’) 
(A perfect reconstruction filter bank which is not a DWT). 
Consider mo(w) = —4 + $e” +e7 7 and mo(w) = —e” + $ + 3c”. 

(a) Show that mo(0) =770(0)=1, mo(m) = mo(a) =0, and that 

Mo(w)mo(w) + Mo(w + 7)MG6(w+7) = 1. 

(b) Hence we are facing a perfect reconstruction filter bank. Verify that we 
have, for the analysis filters ho and h; and for the synthesis filters go and 
g1: 

(ho[-1], hol0], holt]) = (2¥2, 2, v3), (gol0), galt], gol21) = (-, 2, v3) 


(Ai[—1], ha [0], ha (1) = (91[0], 91 [1], 91 [2]) = 
(h4 [0], Ai [1], hi [2]) = (gi[-¥], gi [0], 91 (1) = 
(--2,-2, v2) (-32, 2, v2) 


c) Let Mo be the matrix of the operator Po restricted to trigonometric poly- 
g 
nomials of degree < 2. Show that 


S 

oO 

I 

| 

= 

w | 
eS 


a 
ONI-ENIF CO © 
| 
= 


oo 
SO OnNIHNIF © 
[an} | 


Show that the trigonometric polynomial fo(w) = 1—4cosw is “the” solution 
of the equation Po fo = fo. Deduce that the identity 
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“ 7 Ww 
Ho) = Tn (2) 
k=1 


cannot define a valid (i.e. square integrable) scaling function y = y(t) (else 
a(w) = Vue | Pw + 2ak) |? would be proportional to fo(w).) 

(d) Let Mo be the matrix of the operator Po restricted to trigonometric poly- 
nomials of degree < 2. Show that 


I 
aS 
oo 


Zr 
° 
| 
| 
w 
ONIFNIF © © 
“I 


NI 
| 
w 


—3 


oo 
SO Oninvir O 
| 
ww 


0 
Show that the trigonometric polynomial fo(w) = 1 -12cosw is “the” solution 
of the equation Po fo = fo. 
Deduce that the identity 


cannot define a valid scaling function ¢ = ((t). 

A final remark: The simplicity of our example (two impulse responses of length 
3) demanded the sacrifice of symmetry: otherwise, the lengths of the two initial 
impulse responses should differ by an odd multiple of 2. 


(The DWT 5/3 spline) 
Consider 


iw 


m (w) —_ 1iyw 1 1 iw jl 1 iw | 3 A 1 iw 1 -2isw 
oe sae 8 Toca eer 8 
(a) Verify that  mo(0) = mo(0)=1 + mo(m) = mo(7) =0 and that 


+ie 
4 


mo(w)m6(w) + mo(w + 7)mMo(w +7) = 1. 


The perfect reconstruction filter bank thus defined is already familiar to us. 
We want to verify that we actually have a DWT associated with a couple 
of biorthogonal wavelet bases. 

(b) Let Mo be the matrix of the operator Po on the trigonometric polynomials 
of degree < 2. Show that 


g0000 
311 gq9Q 
Mies | Pier 
a ee a 
00535 
0000¢ 


Find “the” fixed point fo = fo(w) of the operator Pp and verify that it is 
strictly positive. 
[Answer: fo(w) = 2+ cosw] 
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(c) Let Mo be the matrix of the operator Py on the trigonometric polynomials 
of degree < 4. Show that 


a00000000 
cba000000 
edcba0000 
cdedcbaO00 
Myp=]abcdedcba 
O0abcdedc 
0O000abcde 
000000abe 
00000000a 


with 4 
Bagh Pgs ge oe gee: 
Find “the” fixed point fo = fo(w) of the operator Py and verify that it is 
strictly positive. 
[Answer: fo(w) = 308 — 201 cosw + 36 cos 2w + cos 3w] 

(d) Conclude by the theorem of Cohen—Daubechies—Feauveau: The filter bank 
given by mo(w) and mo(w) (i.e. by the low-pass synthesis filter go and by 
the low-pass analysis filter ho) is a DWT associated with a biorthogonally 
coupled multi-resolution analysis . 


(7) Consider the perfect reconstruction filter bank given by*® 


1 2iw 1 iw 3 1 iw 1 —2iw 


ole Sage ge he ge OF 
m, (w) 3 ev 3 e7iw | 73 aie | 17 jl 73 Piao 3 mcd 3 enw 
? 280 56 ' 280° | 28 ' 280 56 280 : 


(a) Show, using the Fixed Point Criterion, that we get a DWT associated with 
a couple of biorthogonal wavelet bases. 

(b) Trace the approximations s3 of y = y(t) and §3 of ¢ = ¢(t) by the subdi- 
vision algorithm.°° 


[Help For (a): 

Concerning Mo, the 9 x 9 matrix of the operator Po on the trigonometric poly- 
nomials of degree < 4, we are in the same situation as in part (c) of the preceding 
exercise, now with a = sau b= a c= sau d= sa. e= a 

Observing that our linear system is invariant by inversion of the order of the 
variables — and that the two outer equations are trivial — we know that we can 
search for a solution of the form (0, X3, X2, X1, Xo, X1, X2, X3,0). 

The final triangular system: Xo —10X1 —199X2 +110X3 = 0 

90X1 —396X2 =0 
X_, —210X3 = 0 


49 Cf. exercise (2) of the initial section on design procedures; we deal with the star 
in the family of Burt filters. 

5° Cf. the exercises at the end of the section “The Filter Banks Which are Discrete 
Wavelet Transforms” . 
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s3 for go of the DWT 7/5 Burt. 


This yields our (“unique”) strictly positive polynomial: fo(w) = 25460 + 
924 cos w + 210 cos 2w + cos 3w (corresponding to X3 = 4). 

As to Mo, the 13x 13 matrix of the operator P, on the trigonometric polynomials 
of degree < 6, it now depends — by formal analogy with the preceding one — on the 
7 parameters 


_ 18 ,_ 180 426 
2802’ 2802’ 2802’ 


g=—8420 5 __ AIS. _ 45440 | _ 80052 
2802’ 2802’ 802° 9~ 9802" 


a 
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83 for ho of the DWT 7/5 Burt. 


Once more, we can search, for the homogeneous system defined by Mo — lis, a 
symmetric solution (0, X5, X4, X3, X2, X1, Xo, X1, X2, X3, X4, X5,0). This leads us 
to the following reduced system: 


—39110X5 +9X4 =0 
—3210X5 —39413X4 +90X3 +9X2 =0 
22720X5 —209X4 —42410X3 —213X2 +90.X1 +9Xo = 0 


22720X5 +40026X4 +22720X3 —39400X2 —3120X1 —213X0 = 0 
—3210X5  —200X4 +22810X3 +39813X2 —19690X1 —209Xo = 0 
90X5  —213X4 —3210X3 —209X2 +22720X1 +413X0 = 0 


The solutions are the scalar multiples of the vector (Fo, F,, Fo, F3, Fu, Fs) given 
by: 
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Fo = 5445171178000580 
F, = —99025982875717 
Fy = —20991909243300 


Fy= 1050836279598 
i= —21544955910 
Ps = —550881 


“The” fixed point fo = fo (w) of the operator Po will then be 
fo(w) = Fy + 2F, cosw + 2F) cos 2w + 2F3 cos 3w + 2F1 cos 4w + 2F5 cos 5w. 


It is immediate that fo = fo(w) is strictly positive (cf. the size of Fp!).] 


Let us sum up: We just have answered the first question of the designer for 
biorthogonal wavelet bases: 

How can one decide if a perfect reconstruction filter bank is actually a Discrete 
Wavelet Transform associated with a couple of biorthogonal wavelet bases? 

There remains the second question: 

Which are the criteria for the choice of the appropriate factorization of 

R(x) = (Mt) Ld -2)", M=L+L, into 

R(a) = r(x)F (2), ae 
mMo(w) = (cos $)?” - r(cosw) = yp >, golkle 

85) 


™mo(w) = (co 2b -F(cosw) = a So, ho[—kje 


and what do the multiplicities N = 2D and N = 2D mean? 

Let us first look at the information encapsulated in the value of N = 2L (and 
symmetrically that of N = 2L). 

We shall begin with some preparations. 


—ikw 


Recall A function ~ = 7(t)>! has p zero moments if 


+00 
/ t*a)(t)dt = 0 for 0O<k<p-1 


co 


(hence w is orthogonal to every polynomial of degree < p — 1) 


Exercises 


We shall work in the setting of a coupled multi-resolution analysis, with our notations 
introduced above. The considered filter bank is, clearly, of finite type. 


(1) Show that the following three statements are equivalent: 


(a) The analysis mother wavelet ~ = w(t) has p zero moments. 


(b) = W(w) is zero for w = 0, as well as its p— 1 first higher derivatives. 
(c) go(w) and its p — 1 first higher derivatives are zero for w = 7, i.e. mo(w) 
admits a root to the order > pinw=T. 


°! We shall tacitly suppose it to be square integrable and of compact support. 
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[Hetp: (a) = (b): $0) = SS (-it)*b@at 


(b) @ (c): bQw) = m1 (w) p(w) = mo(x —w)p(w) (and g(0) # 0) 
(2) In the situation of the preceding exercise, show that 
w = w(t) has p zero moments — > the high-pass analysis filter hi = (hi(k]) 
annihilates, for all polynomials q = q(X) of degree < p— 1, the sequence of the 
sample values q = (q[k])xez. 
(hi is thus a kind of “discrete derivative to the order p”). 


[Help: m\"(0) = Dyen k"hilk] OSn<p-]] 


Let us sum up: At first sight, we note that the multiplicity N = 2L of the root in 
w = 7 for the trigonometric polynomial mo(w) reflects the annihilation degree on 
very regular data by the analysis tools » = u(t) and hy = (hi[k]). 

On the one hand, for very regular (i.e. locally almost polynomial) f = f(t), the 
wavelet coefficients (f, pm) will be very small in absolute value (for fine scales), 
since then the Taylor polynomials of f will be annihilated according to the number 
of zero moments of w. 

On the other hand, on digital data, the high-pass analysis filter hi = (hi[k]) 
will annihilate all “polynomial regularities” of degree < N — 1. But there is also a 
dual aspect, concerning 7 = W(t), the synthesis mother wavelet and go = (go[k]), 
the low-pass synthesis filter. 

The key result of the affair is the following (A.Cohen, I-Daubechies, 
J.C.Feauveau): 

Let » = u(t) and % = W(t) be the (synthesis and analysis) mother wavelets 
defined — recall! — from the scaling functions y = y(t) and ¢ = @(t) by 


v(t) =V2-S° glk + lye(2t—k) and h(t) = V2- S~ ha [-k — 1] G(2t — b) 


TE (be, BO) = bin m/Sk,m!s then we have: 

w~ = w(t) m times continuously differentiable = mo(w) is divisible by 
(1 telwymtt, 

w = y(t) m times continuously differentiable = >  mo(w) is divisible by 
(1 4 re aie 


The argument is the following: The biorthogonality and the differentiability to 
the order p — 1 of one of the wavelets imply that the other has p zero moments. 
Then, one concludes by means of the exercise (1) above. We insist: The regularity 
of w = v(t) (to be continuously differentiable to the order m) ”makes pressure” 
on the multiplicity N of the root w = a of the trigonometric polynomial mo(w): 
N>m-+l1. 

There remains the question: Do we also have the inverse effect, i.e. does an 
increasing of N make increase the regularity of w = w(t)? 

The answer is yes, but somehow at low voice. More precisely: First, since the 
degree of regularity of 7 = W(t) is the same as that of y = y(t), we only need discuss 
the later. 
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In order to obtain that y = y(t) is continuously differentiable to the order m, 
we need that | G(w) | is faster decreasing than | w |~""~* (we need that w’"¢(w) is 
summable). 

Recall: In the antechamber of the problem, the situation was like that: Beginning 
with the decompositions 


mmo(w) = (472) p(w) and stow) = (48) gow) 
(aS) =) 


and with the product formulas 

ow) =T1E, mo (He) and Glu) = TIP, to (4), 
the validity of y = y(t) and of ¢ = ¢(t) with the biorthogonality as a supplement) 
was guaranteed by 


Nes ipa B: ath BS ae een 
2 w€[0,27] 
N > + log, B with B= sup | pow) | 


w€[0,27] 


A more refined version was obtained when replacing the term log, B by the 
critical exponent 6 of mo(w) (and the term log, B by the critical exponent ( of 


mo(w)). 
If we were to introduce exigencies of regularity to the order m and m, respec- 
tively, we should await conditions of the type 


N>m+8, 


N>m+ 8. 

And actually these are the correct constraints. 

From another perspective, i.e. by asymptotical arguments, we shall have N ~ 5m 
(hence the regularity of » = y(t) will be, in general, of considerable influence on the 
number of zero moments of 7 = 7)(t)). 

We finally arrive at the question: What does the regularity of the scaling function 
yp = y(t) (and of the mother wavelet ~ = w(t)) mean for the quality of the synthesis 
filter bank of the associated DWT? 

In order to well understand this point, recall the the subdivision algorithm which 
starts with go = (go[k]) (or with gi = (gi[k]), approximating y = y(t) (and ~ = 
w(t)). More precisely: The subdivision algorithm, applied in n steps, starts with the 
sequence yo Tk = 60,n and it synthesizes it n times — neglecting, at every step, the 
high-pass component: all details are uniformly set to zero. 

After n steps, we obtain yk] =Yn (4) keZ. 

We underline: The result of a “pure” synthesis of n steps of the unit impulse is 
the uniform approximation to the order n of the scaling function y = y(t). 

This manner to interpret the subdivision algorithm has a rather surprising con- 
sequence: 


The scaling function y = y(t) describes the “profile of error propagation” 
in pure synthesis (i.e. with all high-pass details set to zero). 
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More concretely: Consider a finite support sequence oe he a (ys [k]), which we 
aim to synthesize in n steps. Admit an error of value € for k = 0. 
The error sequence: a = ye" +e-00,.. After n synthesis steps (“without 


details”) we obtain 


k 
AUK] = WK] + e-em (=) 


Thus the local error of value € at k = 0 has been propagated to an (almost global) 


error (€- en (2) pen 


Example The DWT 5/3 spline (mo(w) = fe” + 5 + te"). 

1— #l for —a"<k <2" 

0 else 

Here, the propagation of a local error at k = 0 is linearly decreasing, then zero. 


We saw, in an earlier exercise: Yn (+) = { 


Exercise 


Show that a local error of ¢ at k = ko ts propagated as follows: 


k — ko 
Qn 


2) =y™ te Sign => 2K] = yO [k] + €- Onl 


e 


Now, we are ready to discuss the importance of the regularity of y = y(t) for 
the stability of the synthesis filter bank of the considered DWT. Since we suppose 
Zo = (gol[k]) of finite (odd) length and symmetric, the synthesis scaling function 
y = y(t) will be even and of finite support. If y = y(t) is differentiable to the order 
m,m > 1, then y = y(t) will admit horizontal tangents to the order m at the 
boundaries of its support. In other words: The more y = y(t) is regular, the more 
its profile (its graph) will be flattened towards the boundary of the support. 

A great regularity thus means (more or less) a guaranty of localized propagation 
of a local error. 

It is in this sense that the regularity of ¢ = y(t) implies the stability of low-pass 
synthesis. But the regularity of y = y(t) is inherited by ~ = W(t); hence, by the 
same arguments (via subdivision towards 7 = 7(t)), we see that the stability of 
high-pass synthesis (on the details) is equally guaranteed. 

Let us sum up. The regularity of y = y(t) (hence also of ~ = w(t)) guarantees the 
stability of the synthesis filter bank, in the sense made precise above. It is correlated 
— via the multiplicity N = 2D of the root w = a of the trigonometric polynomial 
mo(w) = 5 G0(w) — with the number of zero moments of 7(t), i.e. of the analysis 
mother wavelet. This number indicates to what degree polynomial sample values do 
not leave a trace as details (i.e. in high-pass). 

But the situation is symmetric. Only, a great multiplicity N = 2L of the root 
w = 7 of the trigonometric polynomial mo(w) = Ssho(—w), which would have the 
discussed effect on the regularity of 6 = ¢(t) and on the number of zero moments 
of w = w(t), is (conceptually) difficult to justify. 

This finally leads us to the criteria of optimal factorization of R(x) = r(x) -F(2). 
The first viewpoint is the following: Everything for the regularity of » = y(t) (and 
the zero moments of w = w(t)). Hence, we must choose N = 2 relatively large, and 
put r(x) = 1, ie. F(x) = R(x) = peg Cree 1(1—2)" with M=L+L. 


n=0 n or 
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Note that a priori this does not impose any constraint on the value N = 2L; 
but in practice, the lengths of the considered impulse responses should be minimally 
distinct®?; hence N should be small. 

We thus obtain the DWT splines. The second viewpoint is a kind of tribute to 
orthonormality: Everything for a similarity between y = y(t) and ¢ = ¢(t). We 
look for similar profiles (of the graphs). The design strategy is then the following: 

We shall choose N = N (i.e. L = L). Hence the degree M — 1 of R(2) is odd; 
we factorize R(x) as equitable as possible: the factor r(x) will have degree “ — 1, 
the factor 7(x) will have degree ¥. 

We thus obtain the DWT CDF**. The two viewpoints are accepted. We are 
already familiar with the two most elementary members of the two families: On the 
one hand, the DWT 5/3 spline, on the other hand, the DWT 9/7 CDF. 


Investiture of the DWT 5/3 spline and of the DWT 9/7 CDF 


First, we shall be concerned with the DWT 5/3 spline. As we pointed out, at the 

end of the preceding paragraph, the family of Discrete Wavelet Transforms spline 

comes up by the choice mo(w) = (cos $)“%, N = 2L > 2. In this case, we will have 
: N 

(w) = T]2, (cos sar j= (=) = sinc ()” But: sinc (+) is the Fourier 


}; Hence, for N = 2L > 2, p = ¢(t) will be the 


transform of the function I ae 


N — 1-times iterated convolution product T2,4) ok I-24) (w factors 4.41) ; 
In this case, y = y(t) will have its support in [—L, L], will be a polynomial of 
degree N — 1 on every interval [K,k +1], —L< K < L—1, and will be (N — 2) 
times differentiable at the (integer) nodes of interpolation. In other words: y = y(t) 
will be a B-spline of order N. 
Now consider the simplest case: N = 2, i.e. LD = 1. 
2w _1 Tei oie 


First: mo(w) = cos” $ = 5(14+ cosw) = 5e ae 


Then: (w) = (sine (#))° 
ce S18 ps1 
= 


iw 


e 


Hence: y(t) =T_11)*T_aaj)= 0 else 


For mo(w), the options vary according to the choice of N = 2L. We shall choose 
the simplest case: L = 1, i.e. N = 2. 
M=L+L=2, hence R(x) = Q (exe! - x)) is of degree 1: 


R(x) =14+2-5(1—«) = 2-2. This gives: 


To (w) = cos” = - #(cosw) = cos” = - R(cosw) = cos” ae (2 — cosw) 
1 


2 (Fev +54 56%) (2 1 iw se) = 1 iw ; 1 iw 


4 2 #4 2 8 4 
oe oh 1 -iw = 1 —2iw 
4 A 8 


We have indeed recovered our DWT 5/3 spline. 


52 We note that our hypotheses (odd length and symmetry for the impulse responses) 
force these lengths to differ by an odd multiple of 2. 
°3 CDF = Cohen—Daubechies-Feauveau 
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Note that in our spectral formalism, the impulse responses go and ho would have 
irrational coefficients (the coefficients of mo(w) and of mo(w) are multiplied by V2). 
But for practice applications one renormalizes: 


So -— V2-g0, 
ho +> 3° ho 
(or go ——> Fi ‘Zo, ho fo J2 ‘ ho). 


Exercises 


(1) Consider mo(w) and mho(w) — after having fixed N = 2L and N = 2L. 
(a) Show that then mo(w) = ye oo Gane 
(b) In real notation, o(w) = (cos? 4)? - ie (ae) (sin? ¢)*. 
Find the complex notation of the trigonometric polynomial mo (w). 
2) Let us keep mo(w) = +e” +2+4+4e7” and choose now N = 2L = 4. 
4 a7 4 
(a) Find mo(w). 


Answer: mo(w) = 


zee = geo a a —4iw 
(b) For the fans of computation: Show, by means of the Fixed Point Criterion 
applied to the operator Po, that the product formula for G(w) indeed gives a 
valid analysis scaling function y(t) (and that we hence obtain biorthogonal 
wavelet bases). 
(Attention: In principle, we have to triangulate a 17 x 17 linear system the first 
and the last row of which are trivial and which is invariant with respect to order 
inversion of the variables; so we finally only deal with an 8 x 8 system). 
(3) A deviation to the case of odd multiplicities N and N (and of impulse responses 
of even length). 


We have 1s 
Mo(w) = e7'2 (cos 2)N ei 
Mo(w) = e7"3 (cos $)” : nies a) (sin? 2)* 
with M = 4(N +N) 

(a) Choose N = 3 and N = 1. 
Show that then mo(w) = ge“ + 3+ 3e 4 de ie 


mo(w) = —fel” + $+ 37! ze 
(b) Show, by means of the Fixed Point Criterion, that ¢ = G(t) is not valid (P(t) 
is not square integrable). 


(c) Now pass to N = N = 3. Show that we obtain 
~ _— 3 3iw _ 9 Q2iw 7 giw | 45 4 45,-iw 7 ,-2iw _ 9 Q—3iw 1 3 Q—diw 
mmo(w) = gae 64° ea 6a 1 G4e 64 64 


54 The factor ei? creates a symmetric situation with respect to t = 3. 
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Once more: We could show, with the help of the Fixed Point Criterion applied 
to the operator Po, that the product formula for G(w) gives a valid analysis 
scaling function y(t) (and that we hence obtain biorthogonal wavelet bases). 
Note that ¢ = (£(t) is valid without satisfying the condition (D) of controlled 
decreasing. 


Now pass to the DWT 9/7 CDF. 


The situation of the designer is the following: He has chosen N = 2L = N = 21, 


and he now must factor 

R(x) = ee (4) a -2)* (with M=L+L=N=N). 

into two factors r(a) and f(a) of degrees L — 1 and L, respectively. 

This gives 

mMo(w) = (cos $)% - r(cosw) 

mo(w) = (cos $)* - F(cosw) 

The first typical representative’ of this family is obtained for L = L = 2, ie. 
for N=N=4. 

Consider R(x) = pee (7”) oe (1—a)* = 8— Ba+ 100? — 32°. 

R(«) admits the following three roots: 


4 S 7 
hi=5+(i5-35) 


heave ne y+ (5 is 
Se NIG SSS NIB | 288 
4 ae a) we ) 
R3 = i 
3 9\15 39 2 \15 * 38 


with S$ = ¥/(350 + 105v‘15). 
We get”® 


= Wa Ry — cosw 
mo(w) (cos =) ( Root ) 


fno() = (cos ¥)’ (7) =) 
2 Ry—1 Rea 


In complex polynomial notation we hence shall have: 


1 3iw 2-Ri aww , 8Ri-T7 ie 3R, — 2 

a=)  wayS=p ~ sR—1- 

Ohi =i 2-Ri aia 1 —3iw 
"32(Ri —1)° 16(Ri —1)° 32(Ri—1)° 


mo (w) = 


Numerical evaluation of these irrational coefficients gives the following approximate 
notation: 


mo(w) = 0.557543526229 + 0.295635881557(e” +e”) 
—0.028771763114(e7 + e~ 7) — 0.045635881557(e"” +e *”), 
Concerning ™mo(w), we shall treat it in the following 


°° For N = 2, we still have the DWT spline. 
°6 The normalizing denominators guarantee mo(0) = 70(0) = 1 
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(a) Show that 


1 5 
(is —1) (Ag =) gta): 


(b) Compute mo(w) in function of Ri, Re and Rs. 


[Answer: mo(w) = os (Ri — ( Co + Ci cosw + C2 cos 2w + C3 cos 38w + C4 cos 4) 


with 
Co = 144 24R2R3 — 16(R2 + Rs) 
C, = 244 32R2R3 — 28(Re + Rs) 
C2 = 164+ 8R2R3 — 16(Re + Rs) 
C3 = 8 — 4(Ro+ Rs) 
C4 = 2 


Numerical evaluation gives here 


Mo (w) = 0.602949018236 + 0.266864118443(e” +e) 


—0.078223266529(e"” + e 7”) — 0.016864118443(e"" + e ) 


+0.026748757411(e"” + e 4”). 


This is the definition of the DWT 9/7 CDF*”. 

We shall not deal with the validity of the scaling functions y = y(t) and of 
~ = Gt). 
Remark on a renormalization (that we already encountered with the DWT 5/3 
spline): 

According to our conventions, mo(w) = PF -Go(w) and mo(w) = B - ho(—w). 

But in practical implementation for JPEG 2000, one prefers to take the co- 
efficients of 7mo(w) as the coordinates of ho®’. This corresponds to a change 
ho +> a - ho, which forces the rescaling go -— J2- go. The renormalized co- 
efficients of go will then be the coefficients of 2- mo(w). 

The renormalized impulse responses hi and g{ are given by 


(hi [0], hy [1], 24 [2], h4[3]) = (0.557543526229, —0.295635881557, —0.028771763114, 
0.045635881557) (91 [0], i [1], 91[2], 91 [3], 91[4]) = 2 - (0.602949018236, 
—0.266864118443, —0.078223266529, 0.016864118443, 0.026748757411). 


°7 9/7 for the lengths of ho and of go. 
°8 One aims at > , holk] = 1. Thus the low-pass analysis filter computes local means. 
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si for go of the DWT 9/7 CDF. 
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so for go of the DWT 9/7 CDF. 
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s3 for go of the DWT 9/7 CDF. 
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$1 for ho of the DWT 9/7 CDF. 
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S2 for ho of the DWT 9/7 CDF. 
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§3 for ho of the DWT 9/7 CDF. 


References 


Digital Signatures Using Reversible Public Key Cryptography for the Financial Ser- 
vices Industry (rDSA). American National Standards Institute X9.31, 1998. 

Digital Signatures Using Reversible Public Key Cryptography for the Financial Ser- 
vices Industry: The Elliptic Curve Digital Signature Algorithm (ECDSA). Amer- 
ican National Standards Institute X9.62, 1998. 

R.E. Blahut. Theory and Practice of Error Control Codes. Addison-Wesley, 1983. 

R.E. Blahut. Digital Transmission of Information. Addison-Wesley, 1990. 

P. Brémaud. An Introduction to Probabilistic Modeling. UTM, Springer, Berlin Hei- 
delberg New York, 1988. 

D.M. Bressoud. Factorization and Primality Testing. Springer, Berlin Heidelberg 
New York, 1989. 

R. Calderbank, I. Daubechies, W. Sweldens, and B. Yeo. Wavelet transforms 
that map integers to integers. Applied and Computational Harmonic Analysis, 
5(3):332-369, July 1998. 

A. Cohen, I. Daubechies, and J.-C. Feauveau. Biorthogonal bases of compactly sup- 
ported wavelets. Communications on Pure and Applied Mathematics, 45(5):485— 
560, June 1992. 

D.A. Cox. Primes of the form x? + ny”. Wiley, 1989. 

J. Daemen and V. Rijmen. The Design of Rijndael. AES - The Advanced Encryption 
Standard. Springer, Berlin Heidelberg New York, 2002. 

I. Daubechies. Orthonormal bases of compactly supported wavelets. Communica- 
tions on Pure and Applied Mathematics, 41:909-996, November 1988. 

W. Diffie and M.E. Hellman. New directions in cryptography. IEEE Transactions 
on Information Theory, IT-22:644-654, 1976. 

T. ElGamal. A public key cryptosystem and a signature scheme based on discrete 
logarithms. IEEE Transactions on Information Theory, IT-31:469—472, 1985. 
D.F. Elliott and K.R. Rao. Fast Transforms. Algorithms, Analyses, Applications. 

Academic, 1982. 

P.M. Farrelle. Recursive Block Coding for Image Data Compression. Springer, Berlin 
Heidelberg New York, 1990. 

D.A. Huffman. A method for the construction of minimum redundancy codes. Pro- 
ceedings of the IRE, 40:1098-1101, 1952. 


436 References 


A.K. Jain. Fundamentals of Digital Image Processing. Prentice-Hall, Englewood 
Cliffs, NJ, 1989. 

C.B. Jones. An efficient coding system for long source sequences. [EEE Transactions 
on Information Theory, IT-27:280—291, 1981. 

N. Koblitz. A Course in Number Theory and Cryptography. Springer, Berlin Heidel- 
berg New York, 1987. 

G. Kraft. A device for quantizing, grouping, and coding amplitude modulated pulses. 
MS Thesis, Department of Electrical Engineering, Massachusetts Institute of 
Technology, Cambridge, MA, 1949. 

G.-L. Lay and H.G. Zimmer. Constructing elliptic curves with given group order 
over large finite fields. In Proceedings of ANTS I, LNCS 877: 250-263, 1994. 

S. Mallat. A theory for multiresolution signal decomposition; the wavelet represen- 
tation. IEEE Transactions on Pattern Analysis and Mathematical Intelligence, 
11(7):674-693, 1989. 

B. McMillan. Two inequalities implied by unique decipherability. JRE Transactions 
on Information Theory, IT-2:115-116, 1956. 

A.J. Menezes. Elliptic Curve Public Key Cryptosystems. Kluwer Academic, Boston 
Dordrecht London, 1993. 

Data Encryption Standard (DES). National Bureau of Standards FIPS Publication 
46, 1977. 

DES modes of operation. National Bureau of Standards FIPS Publication 81, 1980. 

Secure Hash Standard. National Bureau of Standards FIPS Publication 180-1, 1995. 

Digital Signature Standard (DSS). National Bureau of Standards FIPS Publication 
186-2, 2000. 

H. Nyquist. Certain topics in telegraph transmission theory. AIEE Transactions, 
47:617-644, 1928. 

W.B. Pennebaker and J.L. Mitchell. JPEG: Still Image Data Compression Standard. 
Van Nostrand Reinhold, New York, 1992. 

K.R. Rao and P. Yip. Discrete Cosine Transform. Academic, New York, 1990. 

I.S. Reed and G. Solomon. Polynomial codes over certain finite fields. Journal of the 
Society of Industrial Applied Mathematics, 8:300—-304, 1960. 

R.L. Rivest, A. Shamir, and L. Adleman. A method for obtaining digital signa- 
tures and public key cryptosystems. Communications of the ACM, 21:120-126, 
1978. 

J. Rissanen and G.G. Langdon, Jr.. Arithmetic coding. IBM Journal of Research 
and Development, 23:149-162, 1979. 

J. Rissanen and G.G. Langdon, Jr.. Universal modeling and coding. [EEE Transac- 
tions on Information Theory, IT-27:12-23, 1979. 

C.E. Shannon. A mathematical theory of communication. Bell Systems Technical 
Journal, 27:379-423 (Part I), 623-656 (Part II), 1948. 

D. Stinson. Cryptography - Theory and Practice. CRC, Boca Raton, 1995. 

R. Strichartz. A Guide to Distribution Theory and Fourier Transforms. CRC, Boca 
Raton, 1994. 

D.S. Taubman and M.W. Marcellin. JPEG2000. Image Compression Fundamentals, 
Standards and Practice. Kluwer Academic, Boston Dordrecht London, 2002. 

H. Triebel. Theory of Function Spaces. Birkhauser Verlag, Boston, 1992. 

M. Vetterli and J. Kovacevié. Wavelets and Subband Coding. Prentice-Hall, NJ, 
1995. 


References 437 


A.J. Viterbi. Error bounds for convolutional codes and an asymptotically optimum 
decoding algorithm. [EEE Transactions on Information Theory, IT-13:260-269, 
1967. 

J. Whittaker. Interpolatory function theory. Cambridge Tracts in Math. and Math. 
Physics, 33, 1935. 

J. Ziv and A. Lempel. A Universal Algorithm for Sequential Data Compression. 
IEEE Transactions on Information Theory, 23(3):337-343, 1977. 


Index 


Page numbers followed by n indicate foot notes. 


Advanced Encryption Standard, 60-91 
almost everywhere (a.e.), 201 

analysis approximation, 389, 391 
analysis filter bank, 320, 336, 350, 382 
analysis filters, 325, 391, 415 

analysis matrix, 318, 365n, 385n 
arithmetic coding, 32-37, 40 
arithmetic decoder, 37 

autoregressive process, 312 


average code word length, 13, 18-20, 33 


balanced trigonometric polynomials, 
195, 271 

binary notation of a real number, 15 

binary source, 6, 8 

binary tree, 11, 19 

biorthogonal system, 391 

biorthogonal wavelet bases, 410, 411 

block, 17, 24, 31-33, 35-36, 50, 53-54, 
57, 59, 77-78, 89, 91, 101-103, 
105, 108, 119, 188-189, 197, 222, 
224-226, 250, 252 

block encoding, 31, 33 

bounded variation, 199 


catastrophic convolutional code, 244 

Chinese Remainder Theorem, 63, 97, 
124n 

Cholesky algorithm, 293 

chrominance, 22, 301 


cipher key, 50, 57, 102 

ciphertext, 50, 78, 99 

circular matrix, 82, 177 

classgroup, 133, 160, 162 

class invariant, 148, 162 

classnumber, 134, 148, 160 

class polynomial, 132, 148, 156, 160 

CM discriminant, 132-133, 144, 162 

code bitstream, 239, 253 

code word, 33, 222, 236, 238, 247 

code word of an interval, 39 

complex multiplication, 131, 158, 162, 
181 

compressed bit-rate, 20, 27, 339 

convolution, 239-266 

convolution product, circular, 83, 178 

Convolution Theorem, 177, 181, 212, 
234, 268 

convolutional code, 239-266 

coprimality, 64, 244 

covariance matrix, 305 

critical exponent, 402, 422 

cyclic group, 69, 73 

cyclic code, 225 

cyclic left shift, 57, 104 


data compaction, 5-46 

Data Encryption Standard, 50-59 
deconvolution, 314, 317n 
decorrelation, 268, 314, 323 
Dedekind’s 7-function, 162 


440 Index 


Deuring Reduction Theorem, 162 
De Moivre Formula, 173 

density of the prime numbers, 118 
DES scheme, 50-52 
diagonalization, 285, 305 
dictionary, 45, 47 

digital filter, 239-252, 268, 271 


digital image processing, 267, 293, 323 


digital signature, 101-169 

Digital Signature Algorithm, 2, 112 
Digital Signature Standard, 101 
digitization, 171, 172, 377 

dilation, 373, 377, 380 

Dirac, 210, 211 

Dirichlet theorem, 203 

Discrete Cosine Transform, 3, 32, 


274-313 

Discrete Fourier Transform, 172-189, 
227 

discrete logarithm problem, 69, 126, 
163n, 167 


discrete n-periodic signal, 269, 314 
Discrete Sine Transform, 310-311 
discrete source, 5 

Discrete Wavelet Transform, 314—426 
discriminant, 132 

distortion, 171-172, 268 
down-sampling, 318, 385 

DSA, 112-115, 168 

DWT 5/3 spline, 4, 324, 372, 408 
DWT 7/5 Burt, 366 

DWT 9/7 CDF, 324, 424 


ECDSA, 125-169 

eigenvalue, 287, 289 
eigenvector, 179, 311 

Elias encoder, 33, 40 

elliptic curve, 125, 158 

elliptic function, 159 

entropy, 5 

entropy coding, 5-42 

error correcting codes, 221, 239 
error correction, 224, 232, 246 
Euclidean algorithm, 62, 244 
Euler’s totient function, 93 
expanded key array, 86, 88, 102 
expansion table, 54 
expectation, 305 


extremal properties of eigenvalues, 289 


fast exponentiation, 100 
Fast Fourier Transform algorithm, 183 
Fejér theorem, 208 

Fermat, 94, 117 

FFT, 183, 188 

field, 65, 67, 133-134 

filter bank, 314 

filtration, 373, 374, 375 

finite automata, 3, 239 

Fixed Point Criterion, 412, 417, 425 
formal power series, 240, 253 
Fourier coefficients, 198, 202, 216 
Fourier series, 198 

Fourier Transform, 172, 183 
fractional ideal, 133, 138 

frame, 245, 255 

fundamental region, 135, 139, 147 


generator polynomial, 222, 230 
Gibbs’ inequality, 8 
Golay code, 225 


Hamming code, 221 

Hamming distance, 257, 263 

Hasse theorem, 132 

Hermitian matrix, 283 

hexadecimal notation, 77, 102 

high-pass component, 318, 319, 422 

high-pass subband, 324, 386 

Hilbert class field, 160, 161 

Hilbert space, 200, 315, 373 

homogeneous coordinates, 128 

homothetic lattices, 158-160 

horizontally high-pass subband, 332, 
334 

Huffman algorithm, 18, 31 


imaginary quadratic field, 133, 159 
impulse response, 83, 321, 375n, 401 
information bitstream, 255, 265 
information content, 1, 6 
information word, 222, 255 

initial permutation, 50, 53 
interleaved lecture, 318 

interleaved matrix, 330, 356 
interleaved sequence, 324 
interpolation, 271, 374, 424 
invariant linear filter, 270, 316 
irreducible, 65, 127 

isogeny, 163 


Jacobi algorithm, 287, 289 
JPEG, 22, 297, 323 
JPEG 2000, 323 


Karhunen-Loéve Transform, 267, 305, 
309 

key schedule, 57, 86, 103 

KLT, 267, 302, 310 

Kraft’s inequality, 10, 13 

Kronecker product, 296 


Lagrange, 94, 95 

lattice, 133, 160 

Laurent polynomial, 315 
Lay-Zimmer method, 131, 158 
Lebesgue integral, 201, 209 
ifting step, 363 

ifting structure, 362 

inear phase, 325, 388, 410 
ogtable for Rijndael, 89 
ossless data compression, 5, 221 
ossy data compression, 5 
ow-pass component, 319 
ow-pass subband, 324, 386 
uminance, 22, 300, 357 

LZW, 43 


Meggitt decoder, 225 
memoryless source, 8, 9 
message digests, 101 

message padding, 101 
Miller-Rabin primality test, 116 
minimum distance, 222, 246 
minimum weight, 247 
MixColumns, 78, 82 

modular function, 148, 162 
modular group, 134, 147 
mother wavelet, 381, 423 
multiplicatively invertible, 62 
multi-resolution analysis, 373, 379, 401 


nearly prime, 132, 147 
nth roots of unity, 172 
Nyquist-Shannon theorem, 204, 217 


optimal binary prefix code, 29 
orthogonal matrix, 280, 285 
orthogonal projection, 374, 392 
orthogonal transformation, 267, 274 


Index 441 


orthonormal basis, 283, 285, 290 
orthonormal representation, 308 


Parseval identity, 202 

parity-check equations, 228 

passband filtering, 268 

perfect reconstruction, 314, 326, 382, 
393, 401, 403, 410, 420 

piecewise constant approximation, 377 

plaintext, 51, 99, 120 

Plancherel formula, 202, 214, 374 

Pohlig—Hellman systems, 95 

point at infinity, 127, 164 

POLLARD-rho method, 167 

polynomial code, 221 

positive semidefinite, 306 

prefix code, 11 

primitive, 19, 68, 140 

primitive solution, 140, 153 

private key, 98, 113, 163 

probability distribution, 6 

product distribution, 8, 20 

projective curve, 128, 159 

proper representation (of an integer), 
142-143, 145 

public key, 93, 126 

public key system, 98, 114 


QMF filter bank, 400, 401 
quadratic form, 137, 174 
quantization, 354, 357, 380 
quantization table, 300, 301, 304 
quantized coefficients, 24, 301 
quantized scheme, 22, 29 


random vector, 305, 307 

real symmetric matrix, 285, 306 
rDSA, 122 

reduced class polynomial, 132, 148, 156 
reduced ideal, 136 

reduced (symmetric) matrix, 149-150 
reduction of an elliptic curve, 160-162 
Reed-Solomon codes, 221-235 
remainder, 63, 97, 124n 

resolution levels, 340, 345, 377, 385 
reversible transform, 362 

Riesz basis, 389 

Rijndael, 60, 77 

ring of (algebraic) integers, 133 


442 Index 


round key, 50, 77, 85, 103 
round transformation, 51, 78 
RSA, 93, 97, 122 


sampling of order n, 193 

Sampling Theorem, 196, 214, 268 

S-box, 54, 78, 92 

S-boxes, 54-56, 78 

scaling equations, 395, 397, 404 

scaling factor, 6, 9, 376 

scaling function, 373, 381, 422 

scaling operations, 375 

section, 401, 417n 

Secure Hash Algorithm, 2, 101 

SHA-1, 101 

Shannon codes, 13 

Shannon multi-resolution analysis, 379 

ShiftRows, 78, 81 

signature generation, 113, 114, 123, 166, 
168 

signature verification, 113, 114, 164, 169 

sin c function, 209-210, 212, 372-374, 
380, 424 

sinusoid, 213, 215 

Splitting Theorem, 183, 184, 189 

square integrable functions, 201, 213, 
373, 378 

square summable sequences, 200 

standard intervals, 16, 34 

state array, 85, 103 

stationary 1st-order Markov process, 
312 

strong pseudoprime, 116 

subband, 319-320, 324, 326, 335, 364, 
370, 386 

subband transform, 319, 320, 364 

subdivision algorithm, 404, 407 

summable function, 209, 214, 398 

summation formula, 398, 411 

symmetrical cryptosystem, 97 

symmetric extension, 324, 333, 361 

syndrome, 223, 224, 225 

synthesis approximation, 389, 391 

synthesis filter bank, 320, 390, 423 

synthesis filters, 325, 386 

synthesis matrix, 318, 329, 340 


Tchebyshev’s inequality, 307 
tempered distributions, 211 


tensor product, 293, 295 

time series, 172, 198n 

translated notation, 325 

transmission error, 222, 250, 265 

trellis codes, 239 

trigonometric interpolation, 190-197 

trigonometric polynomial, 191 
complex notation, 191, 213, 268 

two channel filter bank, 314-371, 382, 

386 
two dimensional DCT, 293 
two dimensional DWT, 330, 342, 343 


unconditional basis, 389 
Uniformization Theorem, 159 
unit circle, 134, 172, 193 
universal codes, 43 

upper half-plane, 134, 162 
up-sampling, 385n, 401 


vertically high-pass subband, 332, 345, 
347 
Viterbi decoder, 253, 254 


Walsh-Hadamard transform, 284, 296 

wavelet, 3-4, 267-268, 314-316, 323 
324, 326, 362, 366, 368, 372-379, 
381, 383-387, 389, 391-392, 398, 
401, 403-404, 410-412, 416-417, 
420-421, 423-426 

wavelet basis, 381, 384, 391 

Weber functions, 162 

Weierstrass equation, 159, 160 

Weierstrass g-function, 159 

weighted node, 19 

white noise, 312, 313 

Whittaker, 198-199 

Whittaker - Shannon theorem, 198-219 

word, 222-226, 232-233, 235-236, 
238-240, 243, 246-249, 252, 254, 
309, 312, 316, 375, 377, 385, 393, 
399 


zero moment, 420, 421, 422 
zero runlength, 23 
z-transform, 316 


SIGNALS AND COMMUNICATION TECHNOLOGY 


(continued from page ii) 


Chaos-Based Digital 

Communication Systems 

Operating Principles, Analysis Methods, 

and Performance Evalutation 

EC.M. LauandC.K. Tse ISBN 3-540-00602-8 


Adaptive Signal Processing 

Application to Real-World Problems 

J. Benesty and Y. Huang (Eds.) 

ISBN 3-540-00051-8 

Multimedia Information Retrieval 

and Management 

Technological Fundamentals and Applications 
D. Feng, W.C. Siu, and H.J. Zhang (Eds.) 
ISBN 3-540-00244-8 

Structured Cable Systems 

A.B. Semenoy, S.K. Strizhakov, 

and I.R.Suncheley ISBN 3-540-43000-8 


UMTS 

The Physical Layer of the Universal Mobile 
Telecommunications System 

A. Springer and R. Weigel 

ISBN 3-540-42162-9 

Advanced Theory of Signal Detection 

Weak Signal Detection in 

Generalized Obeservations 

I. Song, J. Bae, and S.Y. Kim 

ISBN 3-540-43064-4 

Wireless Internet Access over GSM and UMTS 
M. Taferner and E. Bonek 

ISBN 3-540-42551-9 


